Autonomous vehicle operation feature monitoring and evaluation of effectiveness

ABSTRACT

Methods and systems for monitoring use and determining risks associated with operation of a vehicle having one or more autonomous operation features are provided. According to certain aspects, a virtual log of data regarding performance of the features in a virtual test environment may be recorded during operation of the vehicle. This may include information regarding the vehicle, the vehicle environment, use of the autonomous operation features, and/or control decisions made by the features. The control decisions may include evasive maneuvers performed by the vehicle under the control of the features. The performance data in the virtual log may be used to determine risk levels associated with vehicle operation by the autonomous operation features. The risk levels may further be used to adjust an insurance policy associated with the vehicle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of, and claims the benefit of, U.S.patent application Ser. No. 15/410,192, entitled “Autonomous VehicleOperation Feature Monitoring and Evaluation of Effectiveness” and filedJan. 19, 2017, which is a divisional application divided from andclaiming priority to U.S. patent application Ser. No. 14/713,266 (filedMay 15, 2015), which claims the benefit of U.S. Provisional ApplicationNo. 62/000,878 (filed May 20, 2014); U.S. Provisional Application No.62/018,169 (filed Jun. 27, 2014); U.S. Provisional Application No.62/035,660 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,669 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,723 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,729 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,769 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,780 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,832 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,859 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,867 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,878 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,980 (filed Aug. 11, 2014); U.S. Provisional Application No.62/035,983 (filed Aug. 11, 2014); U.S. Provisional Application No.62/036,090 (filed Aug. 11, 2014); U.S. Provisional Application No.62/047,307 (filed Sep. 8, 2014); and U.S. Provisional Application No.62/056,893 (filed Sep. 29, 2014). The entirety of each of the foregoingapplications is incorporated by reference herein.

Additionally, the present application is related to co-pending U.S.patent application Ser. No. 14/713,271 (filed May 15, 2015); co-pendingU.S. patent application Ser. No. 14/951,774 (filed Nov. 25, 2015);co-pending U.S. patent application Ser. No. 14/713,184 (filed May 15,2015); co-pending U.S. patent application Ser. No. 14/713,188 (filed May15, 2015); co-pending Ser. No. 14/978,266 (filed Dec. 22, 2015);co-pending U.S. patent application Ser. No. 14/713,194 (filed May 15,2015); co-pending U.S. patent application Ser. No. 14/713,201 (filed May15, 2015); co-pending U.S. patent application Ser. No. 14/713,206 (filedMay 15, 2015); co-pending U.S. patent application Ser. No. 14/713,214(filed May 15, 2015); co-pending U.S. patent application Ser. No.14/713,217 (filed May 15, 2015); co-pending U.S. patent application Ser.No. 14/713,223 (filed May 15, 2015); co-pending U.S. patent applicationSer. No. 14/713,226 (filed May 15, 2015); co-pending U.S. patentapplication Ser. No. 14/713,230 (filed May 15, 2015); co-pending U.S.patent application Ser. No. 14/713,237 (filed May 15, 2015); co-pendingU.S. patent application Ser. No. 14/713,240 (filed May 15, 2015);co-pending U.S. patent application Ser. No. 14/713,244 (May 15, 2015);co-pending U.S. patent application Ser. No. 14/713,249 (filed May 15,2015); co-pending U.S. patent application Ser. No. 14/713,254 (filed May15, 2015); co-pending U.S. patent application Ser. No. 14/951,803 (filedNov. 25, 2015); co-pending U.S. patent application Ser. No. 14/713,261(filed May 15, 2015); and co-pending U.S. patent application Ser. No.14/951,798 (filed Nov. 25, 2015).

FIELD

The present disclosure generally relates to systems and methods fordetermining risk, pricing, and offering vehicle insurance policies,specifically vehicle insurance policies where vehicle operation ispartially or fully automated.

BACKGROUND

Vehicle or automobile insurance exists to provide financial protectionagainst physical damage and/or bodily injury resulting from trafficaccidents and against liability that could arise therefrom. Typically, acustomer purchases a vehicle insurance policy for a policy rate having aspecified term. In exchange for payments from the insured customer, theinsurer pays for damages to the insured which are caused by coveredperils, acts, or events as specified by the language of the insurancepolicy. The payments from the insured are generally referred to as“premiums,” and typically are paid on behalf of the insured over time atperiodic intervals. An insurance policy may remain “in-force” whilepremium payments are made during the term or length of coverage of thepolicy as indicated in the policy. An insurance policy may “lapse” (orhave a status or state of “lapsed”), for example, when premium paymentsare not being paid or if the insured or the insurer cancels the policy.

Premiums may be typically determined based upon a selected level ofinsurance coverage, location of vehicle operation, vehicle model, andcharacteristics or demographics of the vehicle operator. Thecharacteristics of a vehicle operator that affect premiums may includeage, years operating vehicles of the same class, prior incidentsinvolving vehicle operation, and losses reported by the vehicle operatorto the insurer or a previous insurer. Past and current premiumdetermination methods do not, however, account for use of autonomousvehicle operating features. The present embodiments may, inter alia,alleviate this and/or other drawbacks associated with conventionaltechniques.

BRIEF SUMMARY

The present embodiments may be related to autonomous or semi-autonomousvehicle functionality, including driverless operation, accidentavoidance, or collision warning systems. These autonomous vehicleoperation features may either assist the vehicle operator to more safelyor efficiently operate a vehicle or may take full control of vehicleoperation under some or all circumstances. The present embodiments mayalso facilitate risk assessment and premium determination for vehicleinsurance policies covering vehicles with autonomous operation features.

In accordance with the described embodiments, the disclosure hereingenerally addresses systems and methods for evaluating risk associatedwith an autonomous or semi-autonomous vehicle technology for a vehicle.An on-board computer or mobile device may monitor and record vehicleoperating data, including information regarding the decisions made bythe autonomous or semi-autonomous vehicle technology, regardless ofwhether the decisions are actually used to control the vehicle. A servermay receive the operating data and may process this data to determinerisk levels associated with operation of the vehicle.

In one aspect, a computer-implemented method for evaluating riskassociated with an autonomous or semi-autonomous vehicle technology fora vehicle may be provided. The computer-implemented method may includecollecting a virtual log of actual driving performance of the autonomousor semi-autonomous functionality from the vehicle, determining aninsurance-based risk associated with the autonomous or semi-autonomousfunctionality based upon, at least in part (i.e., wholly or partially),analysis of the virtual log of actual driving performance of theautonomous or semi-autonomous functionality, determining an adjustmentto the insurance policy based upon, at least in part (i.e., wholly orpartially), the determined insurance-based risk associated with theautonomous or semi-autonomous functionality, and/or causing theadjustment to the insurance policy to be implemented. The adjustment tothe insurance policy may include one or more of generating, updating, oradjusting one or more of the following: a premium, a rate, a discount, areward, a deductible, or a limit. Additionally, or alternatively, theinsurance-based risk may include a risk of the vehicle being involved ina vehicle accident. The method may include additional, fewer, oralternate actions, including those discussed elsewhere herein.

In another aspect, a computer system for evaluating risk associated withan autonomous or semi-autonomous vehicle technology for a vehicle may beprovided. The computer system may include one or more processors, one ormore communication modules adapted to communicate data, and anon-transitory program memory coupled to the one or more processors andstoring executable instructions. The executable instruction may, whenexecuted by the one or more processors, cause the computer system tocollect a virtual log of actual driving performance of the autonomous orsemi-autonomous functionality from the vehicle via the one or morecommunication modules, determine an insurance-based risk associated withthe autonomous or semi-autonomous functionality based upon analysis ofthe virtual log of actual driving performance of the autonomous orsemi-autonomous functionality, determine an adjustment to the insurancepolicy based upon the determined insurance-based risk associated withthe autonomous or semi-autonomous functionality, and/or cause theadjustment to the insurance policy to be implemented. The adjustment tothe insurance policy may include one or more of generating, updating, oradjusting one or more of the following: a premium, a rate, a discount, areward, a deductible, or a limit. Additionally, or alternatively, theinsurance-based risk may include a risk of the vehicle being involved ina vehicle accident. The system may include additional, fewer, oralternate actions, including those discussed elsewhere herein.

In yet another aspect, a tangible, non-transitory computer-readablemedium storing instructions for evaluating risk associated with anautonomous or semi-autonomous vehicle technology for a vehicle may beprovided. The instructions may, when executed by at least one processorof a computer system, cause the computer system to collect a virtual logof actual driving performance of the autonomous or semi-autonomousfunctionality from the vehicle, determine an insurance-based riskassociated with the autonomous or semi-autonomous functionality basedupon analysis of the virtual log of actual driving performance of theautonomous or semi-autonomous functionality, determine an adjustment tothe insurance policy based upon the determined insurance-based riskassociated with the autonomous or semi-autonomous functionality, and/orcause the adjustment to the insurance policy to be implemented. Theadjustment to the insurance policy may include one or more ofgenerating, updating, or adjusting one or more of the following: apremium, a rate, a discount, a reward, a deductible, or a limit.Additionally, or alternatively, the insurance-based risk may include arisk of the vehicle being involved in a vehicle accident. Theinstructions of the computer-readable medium may include additional,fewer, or alternate actions, including those discussed elsewhere herein.

In some embodiments, determining the insurance-based risk may be basedat least in part upon analysis of one or more of the following indicatedby the virtual log: (1) decisions made by the autonomous orsemi-autonomous functionality; or (2) the operating conditions underwhich the autonomous or semi-autonomous functionality made decisions. Infurther embodiments, determining the insurance-based risk may be basedat least in part upon analysis of one or more of the following indicatedby the virtual log: (1) evasive maneuvers that the autonomous orsemi-autonomous functionality directed the vehicle to perform; or (2)the operating conditions under which the evasive maneuvers wereperformed. In yet further embodiments, determining the insurance-basedrisk may be based at least in part upon analysis of one or more of thefollowing indicated by the virtual log: (1) how the autonomous orsemi-autonomous functionality responded to driving behavior of otherdrivers operating other vehicles near the vehicle; or (2) evaluating theresponsiveness of the autonomous or semi-autonomous functionality to thedriving behavior of other drivers operating other vehicles near thevehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

Advantages will become more apparent to those skilled in the art fromthe following description of the preferred embodiments which have beenshown and described by way of illustration. As will be realized, thepresent embodiments may be capable of other and different embodiments,and their details are capable of modification in various respects.Accordingly, the drawings and description are to be regarded asillustrative in nature and not as restrictive.

The figures described below depict various aspects of the applications,methods, and systems disclosed herein. It should be understood that eachfigure depicts an embodiment of a particular aspect of the disclosedapplications, systems and methods, and that each of the figures isintended to accord with a possible embodiment thereof. Furthermore,wherever possible, the following description refers to the referencenumerals included in the following figures, in which features depictedin multiple figures are designated with consistent reference numerals.

FIG. 1 illustrates a block diagram of an exemplary computer network, acomputer server, a mobile device, and an on-board computer forimplementing autonomous vehicle operation, monitoring, evaluation, andinsurance processes in accordance with the described embodiments;

FIG. 2 illustrates a block diagram of an exemplary on-board computer ormobile device;

FIG. 3 illustrates a flow diagram of an exemplary autonomous vehicleoperation method in accordance with the presently described embodiments;

FIGS. 4A-B illustrate flow diagrams of an exemplary autonomous vehicleoperation monitoring method in accordance with the presently describedembodiments;

FIG. 5 illustrates a flow diagram of an exemplary autonomous operationfeature evaluation method for determining the effectiveness ofautonomous operation features in accordance with the presently describedembodiments;

FIG. 6 illustrates a flow diagram of an exemplary autonomous operationfeature testing method for presenting test conditions to an autonomousoperation feature and observing and recording responses to the testconditions in accordance with the presently described embodiments;

FIG. 7 illustrates a flow diagram of an exemplary autonomous featureevaluation method for determining the effectiveness of an autonomousoperation feature under a set of environmental conditions, configurationconditions, and settings in accordance with the presently describedembodiments;

FIG. 8 illustrates a flow diagram depicting an exemplary embodiment of afully autonomous vehicle insurance pricing method in accordance with thepresently described embodiments;

FIG. 9 illustrates a flow diagram depicting an exemplary embodiment of apartially autonomous vehicle insurance pricing method in accordance withthe presently described embodiments; and

FIG. 10 illustrates a flow diagram depicting an exemplary embodiment ofan autonomous vehicle insurance pricing method for determining risk andpremiums for vehicle insurance policies covering autonomous vehicleswith autonomous communication features in accordance with the presentlydescribed embodiments; and

FIG. 11 illustrates a flow diagram of an exemplary autonomous operationfeature monitoring and feedback method;

FIG. 12 illustrates a flow diagram of an exemplary autonomous operationfeature monitoring and alert method;

FIG. 13 illustrates a flow diagram of an exemplary fault determinationmethod for determining fault following an accident based upon sensordata and communication data; and

FIG. 14 illustrates a high-level flow diagram of an exemplary autonomousautomobile insurance pricing system.

DETAILED DESCRIPTION

The systems and methods disclosed herein generally relate to evaluating,monitoring, pricing, and processing vehicle insurance policies forvehicles including autonomous (or semi-autonomous) vehicle operationfeatures. The autonomous operation features may take full control of thevehicle under certain conditions, viz. fully autonomous operation, orthe autonomous operation features may assist the vehicle operator inoperating the vehicle, viz. partially autonomous operation. Fullyautonomous operation features may include systems within the vehiclethat pilot the vehicle to a destination with or without a vehicleoperator present (e.g., an operating system for a driverless car).Partially autonomous operation features may assist the vehicle operatorin limited ways (e.g., automatic braking or collision avoidancesystems). The autonomous operation features may affect the risk relatedto operating a vehicle, both individually and/or in combination. Toaccount for these effects on risk, some embodiments evaluate the qualityof each autonomous operation feature and/or combination of features.This may be accomplished by testing the features and combinations incontrolled environments, as well as analyzing the effectiveness of thefeatures in the ordinary course of vehicle operation. New autonomousoperation features may be evaluated based upon controlled testing and/orestimating ordinary-course performance based upon data regarding othersimilar features for which ordinary-course performance is known.

Some autonomous operation features may be adapted for use underparticular conditions, such as city driving or highway driving.Additionally, the vehicle operator may be able to configure settingsrelating to the features or may enable or disable the features at will.Therefore, some embodiments monitor use of the autonomous operationfeatures, which may include the settings or levels of feature use duringvehicle operation. Information obtained by monitoring feature usage maybe used to determine risk levels associated with vehicle operation,either generally or in relation to a vehicle operator. In suchsituations, total risk may be determined by a weighted combination ofthe risk levels associated with operation while autonomous operationfeatures are enabled (with relevant settings) and the risk levelsassociated with operation while autonomous operation features aredisabled. For fully autonomous vehicles, settings or configurationsrelating to vehicle operation may be monitored and used in determiningvehicle operating risk.

Information regarding the risks associated with vehicle operation withand without the autonomous operation features may then be used todetermine risk categories or premiums for a vehicle insurance policycovering a vehicle with autonomous operation features. Risk category orprice may be determined based upon factors relating to the evaluatedeffectiveness of the autonomous vehicle features. The risk or pricedetermination may also include traditional factors, such as location,vehicle type, and level of vehicle use. For fully autonomous vehicles,factors relating to vehicle operators may be excluded entirely. Forpartially autonomous vehicles, factors relating to vehicle operators maybe reduced in proportion to the evaluated effectiveness and monitoredusage levels of the autonomous operation features. For vehicles withautonomous communication features that obtain information from externalsources (e.g., other vehicles or infrastructure), the risk level and/orprice determination may also include an assessment of the availabilityof external sources of information. Location and/or timing of vehicleuse may thus be monitored and/or weighted to determine the riskassociated with operation of the vehicle.

Autonomous Automobile Insurance

The present embodiments may relate to assessing and pricing insurancebased upon autonomous (or semi-autonomous) functionality of a vehicle,and not the human driver. A smart vehicle may maneuver itself withouthuman intervention and/or include sensors, processors, computerinstructions, and/or other components that may perform or direct certainactions conventionally performed by a human driver.

An analysis of how artificial intelligence facilitates avoidingaccidents and/or mitigates the severity of accidents may be used tobuild a database and/or model of risk assessment. After which,automobile insurance risk and/or premiums (as well as insurancediscounts, rewards, and/or points) may be adjusted based upon autonomousor semi-autonomous vehicle functionality, such as by groups ofautonomous or semi-autonomous functionality or individual features. Inone aspect, an evaluation may be performed of how artificialintelligence, and the usage thereof, impacts automobile accidents and/orautomobile insurance claims.

The types of autonomous or semi-autonomous vehicle-related functionalityor technology that may be used with the present embodiments to replacehuman driver actions may include and/or be related to the followingtypes of functionality: (a) fully autonomous (driverless); (b) limiteddriver control; (c) vehicle-to-vehicle (V2V) wireless communication; (d)vehicle-to-infrastructure (and/or vice versa) wireless communication;(e) automatic or semi-automatic steering; (f) automatic orsemi-automatic acceleration; (g) automatic or semi-automatic braking;(h) automatic or semi-automatic blind spot monitoring; (i) automatic orsemi-automatic collision warning; (j) adaptive cruise control; (k)automatic or semi-automatic parking/parking assistance; (l) automatic orsemi-automatic collision preparation (windows roll up, seat adjustsupright, brakes pre-charge, etc.); (m) driver acuity/alertnessmonitoring; (n) pedestrian detection; (o) autonomous or semi-autonomousbackup systems; (p) road mapping systems; (q) software security andanti-hacking measures; (r) theft prevention/automatic return; (s)automatic or semi-automatic driving without occupants; and/or otherfunctionality.

The adjustments to automobile insurance rates or premiums based upon theautonomous or semi-autonomous vehicle-related functionality ortechnology may take into account the impact of such functionality ortechnology on the likelihood of a vehicle accident or collisionoccurring. For instance, a processor may analyze historical accidentinformation and/or test data involving vehicles having autonomous orsemi-autonomous functionality. Factors that may be analyzed and/oraccounted for that are related to insurance risk, accident information,or test data may include (1) point of impact; (2) type of road; (3) timeof day; (4) weather conditions; (5) road construction; (6) type/lengthof trip; (7) vehicle style; (8) level of pedestrian traffic; (9) levelof vehicle congestion; (10) atypical situations (such as manual trafficsignaling); (11) availability of internet connection for the vehicle;and/or other factors. These types of factors may also be weightedaccording to historical accident information, predicted accidents,vehicle trends, test data, and/or other considerations.

In one aspect, the benefit of one or more autonomous or semi-autonomousfunctionalities or capabilities may be determined, weighted, and/orotherwise characterized. For instance, the benefit of certain autonomousor semi-autonomous functionality may be substantially greater in city orcongested traffic, as compared to open road or country driving traffic.Additionally or alternatively, certain autonomous or semi-autonomousfunctionality may only work effectively below a certain speed, i.e.,during city driving or driving in congestion. Other autonomous orsemi-autonomous functionality may operate more effectively on thehighway and away from city traffic, such as cruise control. Furtherindividual autonomous or semi-autonomous functionality may be impactedby weather, such as rain or snow, and/or time of day (day light versusnight). As an example, fully automatic or semi-automatic lane detectionwarnings may be impacted by rain, snow, ice, and/or the amount ofsunlight (all of which may impact the imaging or visibility of lanemarkings painted onto a road surface, and/or road markers or streetsigns).

Automobile insurance premiums, rates, discounts, rewards, refunds,points, etc. may be adjusted based upon the percentage of time orvehicle usage that the vehicle is the driver, i.e., the amount of time aspecific driver uses each type of autonomous (or even semi-autonomous)vehicle functionality. In other words, insurance premiums, discounts,rewards, etc. may be adjusted based upon the percentage of vehicle usageduring which the autonomous or semi-autonomous functionality is in use.For example, automobile insurance risk, premiums, discounts, etc. for anautomobile having one or more autonomous or semi-autonomousfunctionalities may be adjusted and/or set based upon the percentage ofvehicle usage that the one or more individual autonomous orsemi-autonomous vehicle functionalities are in use, anticipated to beused or employed by the driver, and/or otherwise operating.

Such usage information for a particular vehicle may be gathered overtime and/or via remote wireless communication with the vehicle. Oneembodiment may involve a processor on the vehicle, such as within avehicle control system or dashboard, monitoring in real-time whethervehicle autonomous or semi-autonomous functionality is currentlyoperating. Other types of monitoring may be remotely performed, such asvia wireless communication between the vehicle and a remote server, orwireless communication between a vehicle-mounted dedicated device (thatis configured to gather autonomous or semi-autonomous functionalityusage information) and a remote server.

In one embodiment, if the vehicle is currently employing autonomous orsemi-autonomous functionality, the vehicle may send a Vehicle-to-Vehicle(V2V) wireless communication to a nearby vehicle also employing the sameor other type(s) of autonomous or semi-autonomous functionality.

As an example, the V2V wireless communication from the first vehicle tothe second vehicle (following the first vehicle) may indicate that thefirst vehicle is autonomously braking, and the degree to which thevehicle is automatically braking and/or slowing down. In response, thesecond vehicle may also automatically or autonomously brake as well, andthe degree of automatically braking or slowing down of the secondvehicle may be determined to match, or even exceed, that of the firstvehicle. As a result, the second vehicle, traveling directly orindirectly, behind the first vehicle, may autonomously safely break inresponse to the first vehicle autonomously breaking.

As another example, the V2V wireless communication from the firstvehicle to the second vehicle may indicate that the first vehicle isbeginning or about to change lanes or turn. In response, the secondvehicle may autonomously take appropriate action, such as automaticallyslow down, change lanes, turn, maneuver, etc. to avoid the firstvehicle.

As noted above, the present embodiments may include remotely monitoring,in real-time and/or via wireless communication, vehicle autonomous orsemi-autonomous functionality. From such remote monitoring, the presentembodiments may remotely determine that a vehicle accident has occurred.As a result, emergency responders may be informed of the location of thevehicle accident, such as via wireless communication, and/or quicklydispatched to the accident scene.

The present embodiments may also include remotely monitoring, inreal-time or via wireless communication, that vehicle autonomous orsemi-autonomous functionality is, or is not, in use, and/or collectinformation regarding the amount of usage of the autonomous orsemi-autonomous functionality. From such remote monitoring, a remoteserver may remotely send a wireless communication to the vehicle toprompt the human driver to engage one or more specific vehicleautonomous or semi-autonomous functionalities.

Another embodiment may enable a vehicle to wirelessly communicate with atraffic light, railroad crossing, toll both, marker, sign, or otherequipment along the side of a road or highway. As an example, a trafficlight may wirelessly indicate to the vehicle that the traffic light isabout to switch from green to yellow, or from yellow to red. In responseto such an indication remotely received from the traffic light, theautonomous or semi-autonomous vehicle may automatically start to brake,and/or present or issue a warning/alert to the human driver. Afterwhich, the vehicle may wirelessly communicate with the vehiclestraveling behind it that the traffic light is about to change and/orthat the vehicle has started to brake or slow down such that thefollowing vehicles may also automatically brake or slow downaccordingly.

Insurance premiums, rates, ratings, discounts, rewards, special offers,points, programs, refunds, claims, claim amounts, etc. may be adjustedfor, or may otherwise take into account, the foregoing functionalityand/or the other functionality described herein. For instance, insurancepolicies may be updated based upon autonomous or semi-autonomous vehiclefunctionality; V2V wireless communication-based autonomous orsemi-autonomous vehicle functionality; and/or vehicle-to-infrastructureor infrastructure-to-vehicle wireless communication-based autonomous orsemi-autonomous vehicle functionality.

Exemplary Embodiments

Insurance providers may currently develop a set of rating factors basedupon the make, model, and model year of a vehicle. Models with betterloss experience receive lower factors, and thus lower rates. One reasonthat this current rating system cannot be used to assess risk forautonomous technology is that many autonomous features vary for the samemodel. For example, two vehicles of the same model may have differenthardware features for automatic braking, different computer instructionsfor automatic steering, and/or different artificial intelligence systemversions. The current make and model rating may also not account for theextent to which another “driver,” in this case the vehicle itself, iscontrolling the vehicle.

The present embodiments may assess and price insurance risks at least inpart based upon autonomous or semi-autonomous vehicle technology thatreplaces actions of the driver. In a way, the vehicle-related computerinstructions and artificial intelligence may be viewed as a “driver.”

In one computer-implemented method of adjusting or generating aninsurance policy, (1) data may be captured by a processor (such as viawireless communication) to determine the autonomous or semi-autonomoustechnology or functionality associated with a specific vehicle that is,or is to be, covered by insurance; (2) the received data may be comparedby the processor to a stored baseline of vehicle data (such as actualaccident information, and/or autonomous or semi-autonomous vehicletesting data); (3) risk may be identified or assessed by the processorbased upon the specific vehicle's ability to make driving decisionsand/or avoid or mitigate crashes; (4) an insurance policy may beadjusted (or generated or created), or an insurance premium may bedetermined by the processor based upon the risk identified that isassociated with the specific vehicle's autonomous or semi-autonomousability or abilities; and/or (5) the insurance policy and/or premium maybe presented on a display or otherwise provided to the policyholder orpotential customer for their review and/or approval. The method mayinclude additional, fewer, or alternate actions, including thosediscussed below and elsewhere herein.

The method may include evaluating the effectiveness of artificialintelligence and/or vehicle technology in a test environment, and/orusing real driving experience. The identification or assessment of riskperformed by the method (and/or the processor) may be dependent upon theextent of control and decision making that is assumed by the vehicle,rather than the driver.

Additionally or alternatively, the identification or assessment ofinsurance and/or accident-based risk may be dependent upon the abilityof the vehicle to use external information (such as vehicle-to-vehicleand vehicle-to-infrastructure communication) to make driving decisions.The risk assessment may further be dependent upon the availability ofsuch external information. For instance, a vehicle (or vehicle owner)may be associated with a geographical location, such as a large city orurban area, where such external information is readily available viawireless communication. On the other hand, a small town or rural areamay or may not have such external information available.

The information regarding the availability of autonomous orsemi-autonomous vehicle technology, such as a particularfactory-installed hardware and/or software package, version, revision,or update, may be wirelessly transmitted to a remote server foranalysis. The remote server may be associated with an insuranceprovider, vehicle manufacturer, autonomous technology provider, and/orother entity.

The driving experience and/or usage of the autonomous or semi-autonomousvehicle technology may be monitored in real time, small timeframes,and/or periodically to provide feedback to the driver, insuranceprovider, and/or adjust insurance policies or premiums. In oneembodiment, information may be wirelessly transmitted to the insuranceprovider, such as from a transceiver associated with a smart car to aninsurance provider remote server.

Insurance policies, including insurance premiums, discounts, andrewards, may be updated, adjusted, and/or determined based upon hardwareor software functionality, and/or hardware or software upgrades.Insurance policies, including insurance premiums, discounts, etc. mayalso be updated, adjusted, and/or determined based upon the amount ofusage and/or the type(s) of the autonomous or semi-autonomous technologyemployed by the vehicle.

In one embodiment, performance of autonomous driving software and/orsophistication of artificial intelligence may be analyzed for eachvehicle. An automobile insurance premium may be determined by evaluatinghow effectively the vehicle may be able to avoid and/or mitigate crashesand/or the extent to which the driver's control of the vehicle isenhanced or replaced by the vehicle's software and artificialintelligence.

When pricing a vehicle with autonomous driving technology, artificialintelligence capabilities, rather than human decision making, may beevaluated to determine the relative risk of the insurance policy. Thisevaluation may be conducted using multiple techniques. Vehicletechnology may be assessed in a test environment, in which the abilityof the artificial intelligence to detect and avoid potential crashes maybe demonstrated experimentally. For example, this may include avehicle's ability to detect a slow-moving vehicle ahead and/orautomatically apply the brakes to prevent a collision.

Additionally, actual loss experience of the software in question may beanalyzed. Vehicles with superior artificial intelligence and crashavoidance capabilities may experience lower insurance losses in realdriving situations.

Results from both the test environment and/or actual insurance lossesmay be compared to the results of other autonomous software packagesand/or vehicles lacking autonomous driving technology to determine arelative risk factor (or level of risk) for the technology in question.This risk factor (or level of risk) may be applicable to other vehiclesthat utilize the same or similar autonomous operation softwarepackage(s).

Emerging technology, such as new iterations of artificial intelligencesystems, may be priced by combining its individual test environmentassessment with actual losses corresponding to vehicles with similarautonomous operation software packages. The entire vehicle software andartificial intelligence evaluation process may be conducted with respectto various technologies and/or elements that affect driving experience.For example, a fully autonomous vehicle may be evaluated based on itsvehicle-to-vehicle communications. A risk factor could then bedetermined and applied when pricing the vehicle. The driver's past lossexperience and/or other driver risk characteristics may not beconsidered for fully autonomous vehicles, in which all driving decisionsare made by the vehicle's artificial intelligence.

In one embodiment, a separate portion of the automobile insurancepremium may be based explicitly on the artificial intelligencesoftware's driving performance and characteristics. The artificialintelligence pricing model may be combined with traditional methods forsemi-autonomous vehicles. Insurance pricing for fully autonomous, ordriverless, vehicles may be based upon the artificial intelligence modelscore by excluding traditional rating factors that measure riskpresented by the drivers. Evaluation of vehicle software and/orartificial intelligence may be conducted on an aggregate basis or forspecific combinations of technology and/or driving factors or elements(as discussed elsewhere herein). The vehicle software test results maybe combined with actual loss experience to determine relative risk.

Exemplary Autonomous Vehicle Operation System

FIG. 1 illustrates a block diagram of an exemplary autonomous vehicleinsurance system 100 on which the exemplary methods described herein maybe implemented. The high-level architecture includes both hardware andsoftware applications, as well as various data communications channelsfor communicating data between the various hardware and softwarecomponents. The autonomous vehicle insurance system 100 may be roughlydivided into front-end components 102 and back-end components 104. Thefront-end components 102 may obtain information regarding a vehicle 108(e.g., a car, truck, motorcycle, etc.) and the surrounding environment.An on-board computer 114 may utilize this information to operate thevehicle 108 according to an autonomous operation feature or to assistthe vehicle operator in operating the vehicle 108. To monitor thevehicle 108, the front-end components 102 may include one or moresensors 120 installed within the vehicle 108 that may communicate withthe on-board computer 114. The front-end components 102 may furtherprocess the sensor data using the on-board computer 114 or a mobiledevice 110 (e.g., a smart phone, a tablet computer, a special purposecomputing device, etc.) to determine when the vehicle is in operationand information regarding the vehicle. In some embodiments of the system100, the front-end components 102 may communicate with the back-endcomponents 104 via a network 130. Either the on-board computer 114 orthe mobile device 110 may communicate with the back-end components 104via the network 130 to allow the back-end components 104 to recordinformation regarding vehicle usage. The back-end components 104 may useone or more servers 140 to receive data from the front-end components102, determine use and effectiveness of autonomous operation features,determine risk levels or premium price, and/or facilitate purchase orrenewal of an autonomous vehicle insurance policy.

The front-end components 102 may be disposed within or communicativelyconnected to one or more on-board computers 114, which may bepermanently or removably installed in the vehicle 108. The on-boardcomputer 114 may interface with the one or more sensors 120 within thevehicle 108 (e.g., an ignition sensor, an odometer, a system clock, aspeedometer, a tachometer, an accelerometer, a gyroscope, a compass, ageolocation unit, a camera, a distance sensor, etc.), which sensors mayalso be incorporated within or connected to the on-board computer 114.The front end components 102 may further include a communicationcomponent 122 to transmit information to and receive information fromexternal sources, including other vehicles, infrastructure, or theback-end components 104. In some embodiments, the mobile device 110 maysupplement the functions performed by the on-board computer 114described herein by, for example, sending or receiving information toand from the mobile server 140 via the network 130. In otherembodiments, the on-board computer 114 may perform all of the functionsof the mobile device 110 described herein, in which case no mobiledevice 110 may be present in the system 100. Either or both of themobile device 110 or on-board computer 114 may communicate with thenetwork 130 over links 112 and 118, respectively. Additionally, themobile device 110 and on-board computer 114 may communicate with oneanother directly over link 116.

The mobile device 110 may be either a general-use personal computer,cellular phone, smart phone, tablet computer, or a dedicated vehicle usemonitoring device. Although only one mobile device 110 is illustrated,it should be understood that a plurality of mobile devices 110 may beused in some embodiments. The on-board computer 114 may be a general-useon-board computer capable of performing many functions relating tovehicle operation or a dedicated computer for autonomous vehicleoperation. Further, the on-board computer 114 may be installed by themanufacturer of the vehicle 108 or as an aftermarket modification oraddition to the vehicle 108. In some embodiments or under certainconditions, the mobile device 110 or on-board computer 114 may functionas thin-client devices that outsource some or most of the processing tothe server 140.

The sensors 120 may be removably or fixedly installed within the vehicle108 and may be disposed in various arrangements to provide informationto the autonomous operation features. Among the sensors 120 may beincluded one or more of a GPS unit, a radar unit, a LIDAR unit, anultrasonic sensor, an infrared sensor, a camera, an accelerometer, atachometer, or a speedometer. Some of the sensors 120 (e.g., radar,LIDAR, or camera units) may actively or passively scan the vehicleenvironment for obstacles (e.g., other vehicles, buildings, pedestrians,etc.), lane markings, or signs or signals. Other sensors 120 (e.g., GPS,accelerometer, or tachometer units) may provide data for determining thelocation or movement of the vehicle 108. Other sensors 120 may bedirected to the interior or passenger compartment of the vehicle 108,such as cameras, microphones, pressure sensors, thermometers, or similarsensors to monitor the vehicle operator and/or passengers within thevehicle 108. Information generated or received by the sensors 120 may becommunicated to the on-board computer 114 or the mobile device 110 foruse in autonomous vehicle operation.

In some embodiments, the communication component 122 may receiveinformation from external sources, such as other vehicles orinfrastructure. The communication component 122 may also sendinformation regarding the vehicle 108 to external sources. To send andreceive information, the communication component 122 may include atransmitter and a receiver designed to operate according topredetermined specifications, such as the dedicated short-rangecommunication (DSRC) channel, wireless telephony, Wi-Fi, or otherexisting or later-developed communications protocols. The receivedinformation may supplement the data received from the sensors 120 toimplement the autonomous operation features. For example, thecommunication component 122 may receive information that an autonomousvehicle ahead of the vehicle 108 is reducing speed, allowing theadjustments in the autonomous operation of the vehicle 108.

In addition to receiving information from the sensors 120, the on-boardcomputer 114 may directly or indirectly control the operation of thevehicle 108 according to various autonomous operation features. Theautonomous operation features may include software applications ormodules implemented by the on-board computer 114 to control thesteering, braking, or throttle of the vehicle 108. To facilitate suchcontrol, the on-board computer 114 may be communicatively connected tothe controls or components of the vehicle 108 by various electrical orelectromechanical control components (not shown). In embodimentsinvolving fully autonomous vehicles, the vehicle 108 may be operableonly through such control components (not shown). In other embodiments,the control components may be disposed within or supplement othervehicle operator control components (not shown), such as steeringwheels, accelerator or brake pedals, or ignition switches.

In some embodiments, the front-end components 102 communicate with theback-end components 104 via the network 130. The network 130 may be aproprietary network, a secure public internet, a virtual private networkor some other type of network, such as dedicated access lines, plainordinary telephone lines, satellite links, cellular data networks,combinations of these. Where the network 130 comprises the Internet,data communications may take place over the network 130 via an Internetcommunication protocol. The back-end components 104 include one or moreservers 140. Each server 140 may include one or more computer processorsadapted and configured to execute various software applications andcomponents of the autonomous vehicle insurance system 100, in additionto other software applications. The server 140 may further include adatabase 146, which may be adapted to store data related to theoperation of the vehicle 108 and its autonomous operation features. Suchdata might include, for example, dates and times of vehicle use,duration of vehicle use, use and settings of autonomous operationfeatures, speed of the vehicle 108, RPM or other tachometer readings ofthe vehicle 108, lateral and longitudinal acceleration of the vehicle108, incidents or near collisions of the vehicle 108, communicationbetween the autonomous operation features and external sources,environmental conditions of vehicle operation (e.g., weather, traffic,road condition, etc.), errors or failures of autonomous operationfeatures, or other data relating to use of the vehicle 108 and theautonomous operation features, which may be uploaded to the server 140via the network 130. The server 140 may access data stored in thedatabase 146 when executing various functions and tasks associated withthe evaluating feature effectiveness or assessing risk relating to anautonomous vehicle.

Although the autonomous vehicle insurance system 100 is shown to includeone vehicle 108, one mobile device 110, one on-board computer 114, andone server 140, it should be understood that different numbers ofvehicles 108, mobile devices 110, on-board computers 114, and/or servers140 may be utilized. For example, the system 100 may include a pluralityof servers 140 and hundreds of mobile devices 110 or on-board computers114, all of which may be interconnected via the network 130.Furthermore, the database storage or processing performed by the one ormore servers 140 may be distributed among a plurality of servers 140 inan arrangement known as “cloud computing.” This configuration mayprovide various advantages, such as enabling near real-time uploads anddownloads of information as well as periodic uploads and downloads ofinformation. This may in turn support a thin-client embodiment of themobile device 110 or on-board computer 114 discussed herein.

The server 140 may have a controller 155 that is operatively connectedto the database 146 via a link 156. It should be noted that, while notshown, additional databases may be linked to the controller 155 in aknown manner. For example, separate databases may be used for autonomousoperation feature information, vehicle insurance policy information, andvehicle use information. The controller 155 may include a program memory160, a processor 162 (which may be called a microcontroller or amicroprocessor), a random-access memory (RAM) 164, and an input/output(I/O) circuit 166, all of which may be interconnected via anaddress/data bus 165. It should be appreciated that although only onemicroprocessor 162 is shown, the controller 155 may include multiplemicroprocessors 162. Similarly, the memory of the controller 155 mayinclude multiple RAMs 164 and multiple program memories 160. Althoughthe I/O circuit 166 is shown as a single block, it should be appreciatedthat the I/O circuit 166 may include a number of different types of I/Ocircuits. The RAM 164 and program memories 160 may be implemented assemiconductor memories, magnetically readable memories, or opticallyreadable memories, for example. The controller 155 may also beoperatively connected to the network 130 via a link 135.

The server 140 may further include a number of software applicationsstored in a program memory 160. The various software applications on theserver 140 may include an autonomous operation information monitoringapplication 141 for receiving information regarding the vehicle 108 andits autonomous operation features, a feature evaluation application 142for determining the effectiveness of autonomous operation features undervarious conditions, a compatibility evaluation application 143 fordetermining the effectiveness of combinations of autonomous operationfeatures, a risk assessment application 144 for determining a riskcategory associated with an insurance policy covering an autonomousvehicle, and an autonomous vehicle insurance policy purchase application145 for offering and facilitating purchase or renewal of an insurancepolicy covering an autonomous vehicle. The various software applicationsmay be executed on the same computer processor or on different computerprocessors.

FIG. 2 illustrates a block diagram of an exemplary mobile device 110 oran exemplary on-board computer 114 consistent with the system 100. Themobile device 110 or on-board computer 114 may include a display 202, aGPS unit 206, a communication unit 220, an accelerometer 224, one ormore additional sensors (not shown), a user-input device (not shown),and/or, like the server 140, a controller 204. In some embodiments, themobile device 110 and on-board computer 114 may be integrated into asingle device, or either may perform the functions of both. The on-boardcomputer 114 (or mobile device 110) interfaces with the sensors 120 toreceive information regarding the vehicle 108 and its environment, whichinformation is used by the autonomous operation features to operate thevehicle 108.

Similar to the controller 155, the controller 204 may include a programmemory 208, one or more microcontrollers or microprocessors (MP) 210, aRAM 212, and an I/O circuit 216, all of which are interconnected via anaddress/data bus 214. The program memory 208 includes an operatingsystem 226, a data storage 228, a plurality of software applications230, and/or a plurality of software routines 240. The operating system226, for example, may include one of a plurality of general purpose ormobile platforms, such as the Android™, iOS®, or Windows® systems,developed by Google Inc., Apple Inc., and Microsoft Corporation,respectively. Alternatively, the operating system 226 may be a customoperating system designed for autonomous vehicle operation using theon-board computer 114. The data storage 228 may include data such asuser profiles and preferences, application data for the plurality ofapplications 230, routine data for the plurality of routines 240, andother data related to the autonomous operation features. In someembodiments, the controller 204 may also include, or otherwise becommunicatively connected to, other data storage mechanisms (e.g., oneor more hard disk drives, optical storage drives, solid state storagedevices, etc.) that reside within the vehicle 108.

As discussed with reference to the controller 155, it should beappreciated that although FIG. 2 depicts only one microprocessor 210,the controller 204 may include multiple microprocessors 210. Similarly,the memory of the controller 204 may include multiple RAMs 212 andmultiple program memories 208. Although FIG. 2 depicts the I/O circuit216 as a single block, the I/O circuit 216 may include a number ofdifferent types of I/O circuits. The controller 204 may implement theRAMs 212 and the program memories 208 as semiconductor memories,magnetically readable memories, or optically readable memories, forexample.

The one or more processors 210 may be adapted and configured to executeany of one or more of the plurality of software applications 230 or anyone or more of the plurality of software routines 240 residing in theprogram memory 204, in addition to other software applications. One ofthe plurality of applications 230 may be an autonomous vehicle operationapplication 232 that may be implemented as a series of machine-readableinstructions for performing the various tasks associated withimplementing one or more of the autonomous operation features accordingto the autonomous vehicle operation method 300. Another of the pluralityof applications 230 may be an autonomous communication application 234that may be implemented as a series of machine-readable instructions fortransmitting and receiving autonomous operation information to or fromexternal sources via the communication module 220. Still anotherapplication of the plurality of applications 230 may include anautonomous operation monitoring application 236 that may be implementedas a series of machine-readable instructions for sending informationregarding autonomous operation of the vehicle to the server 140 via thenetwork 130.

The plurality of software applications 230 may call various of theplurality of software routines 240 to perform functions relating toautonomous vehicle operation, monitoring, or communication. One of theplurality of software routines 240 may be a configuration routine 242 toreceive settings from the vehicle operator to configure the operatingparameters of an autonomous operation feature. Another of the pluralityof software routines 240 may be a sensor control routine 244 to transmitinstructions to a sensor 120 and receive data from the sensor 120. Stillanother of the plurality of software routines 240 may be an autonomouscontrol routine 246 that performs a type of autonomous control, such ascollision avoidance, lane centering, or speed control. In someembodiments, the autonomous vehicle operation application 232 may causea plurality of autonomous control routines 246 to determine controlactions required for autonomous vehicle operation. Similarly, one of theplurality of software routines 240 may be a monitoring and reportingroutine 248 that transmits information regarding autonomous vehicleoperation to the server 140 via the network 130. Yet another of theplurality of software routines 240 may be an autonomous communicationroutine 250 for receiving and transmitting information between thevehicle 108 and external sources to improve the effectiveness of theautonomous operation features. Any of the plurality of softwareapplications 230 may be designed to operate independently of thesoftware applications 230 or in conjunction with the softwareapplications 230.

When implementing the exemplary autonomous vehicle operation method 300,the controller 204 of the on-board computer 114 may implement theautonomous vehicle operation application 232 to communicate with thesensors 120 to receive information regarding the vehicle 108 and itsenvironment and process that information for autonomous operation of thevehicle 108. In some embodiments including external source communicationvia the communication component 122 or the communication unit 220, thecontroller 204 may further implement the autonomous communicationapplication 234 to receive information for external sources, such asother autonomous vehicles, smart infrastructure (e.g., electronicallycommunicating roadways, traffic signals, or parking structures), orother sources of relevant information (e.g., weather, traffic, localamenities). Some external sources of information may be connected to thecontroller 204 via the network 130, such as the server 140 orinternet-connected third-party databases (not shown). Although theautonomous vehicle operation application 232 and the autonomouscommunication application 234 are shown as two separate applications, itshould be understood that the functions of the autonomous operationfeatures may be combined or separated into any number of the softwareapplications 230 or the software routines 240.

When implementing the autonomous operation feature monitoring andevaluation methods 400-700, the controller 204 may further implement theautonomous operation monitoring application 236 to communicate with theserver 140 to provide information regarding autonomous vehicleoperation. This may include information regarding settings orconfigurations of autonomous operation features, data from the sensors120 regarding the vehicle environment, data from the sensors 120regarding the response of the vehicle 108 to its environment,communications sent or received using the communication component 122 orthe communication unit 220, operating status of the autonomous vehicleoperation application 232 and the autonomous communication application234, or commands sent from the on-board computer 114 to the controlcomponents (not shown) to operate the vehicle 108. The information maybe received and stored by the server 140 implementing the autonomousoperation information monitoring application 141, and the server 140 maythen determine the effectiveness of autonomous operation under variousconditions by implementing the feature evaluation application 142 andthe compatibility evaluation application 143. The effectiveness ofautonomous operation features and the extent of their use may be furtherused to determine risk associated with operation of the autonomousvehicle by the server 140 implementing the risk assessment application144.

In addition to connections to the sensors 120, the mobile device 110 orthe on-board computer 114 may include additional sensors, such as theGPS unit 206 or the accelerometer 224, which may provide informationregarding the vehicle 108 for autonomous operation and other purposes.Furthermore, the communication unit 220 may communicate with otherautonomous vehicles, infrastructure, or other external sources ofinformation to transmit and receive information relating to autonomousvehicle operation. The communication unit 220 may communicate with theexternal sources via the network 130 or via any suitable wirelesscommunication protocol network, such as wireless telephony (e.g., GSM,CDMA, LTE, etc.), Wi-Fi (802.11 standards), WiMAX, Bluetooth, infraredor radio frequency communication, etc. Furthermore, the communicationunit 220 may provide input signals to the controller 204 via the I/Ocircuit 216. The communication unit 220 may also transmit sensor data,device status information, control signals, or other output from thecontroller 204 to one or more external sensors within the vehicle 108,mobile devices 110, on-board computers 114, or servers 140.

The mobile device 110 or the on-board computer 114 may include auser-input device (not shown) for receiving instructions or informationfrom the vehicle operator, such as settings relating to an autonomousoperation feature. The user-input device (not shown) may include a“soft” keyboard that is displayed on the display 202, an externalhardware keyboard communicating via a wired or a wireless connection(e.g., a Bluetooth keyboard), an external mouse, a microphone, or anyother suitable user-input device. The user-input device (not shown) mayalso include a microphone capable of receiving user voice input.

Exemplary Autonomous Vehicle Operation Method

FIG. 3 illustrates a flow diagram of an exemplary autonomous vehicleoperation method 300, which may be implemented by the autonomous vehicleinsurance system 100. The method 300 may begin at block 302 when thecontroller 204 receives a start signal. The start signal may be acommand from the vehicle operator through the user-input device toenable or engage one or more autonomous operation features of thevehicle 108. In some embodiments, the vehicle operator 108 may furtherspecify settings or configuration details for the autonomous operationfeatures. For fully autonomous vehicles, the settings may relate to oneor more destinations, route preferences, fuel efficiency preferences,speed preferences, or other configurable settings relating to theoperation of the vehicle 108. In some embodiments, fully autonomousvehicles may include additional features or settings permitting them tooperate without passengers or vehicle operators within the vehicle. Forexample, a fully autonomous vehicle may receive an instruction to find aparking space within the general vicinity, which the vehicle may dowithout the vehicle operator. The vehicle may then be returned to aselected location by a request from the vehicle operator via a mobiledevice 110 or otherwise. This feature may further be adapted to return afully autonomous vehicle if lost or stolen.

For other autonomous vehicles, the settings may include enabling ordisabling particular autonomous operation features, specifyingthresholds for autonomous operation, specifying warnings or otherinformation to be presented to the vehicle operator, specifyingautonomous communication types to send or receive, specifying conditionsunder which to enable or disable autonomous operation features, orspecifying other constraints on feature operation. For example, avehicle operator may set the maximum speed for an adaptive cruisecontrol feature with automatic lane centering. In some embodiments, thesettings may further include a specification of whether the vehicle 108should be operating as a fully or partially autonomous vehicle. Inembodiments where only one autonomous operation feature is enabled, thestart signal may consist of a request to perform a particular task(e.g., autonomous parking) or to enable a particular feature (e.g.,autonomous braking for collision avoidance). In other embodiments, thestart signal may be generated automatically by the controller 204 basedupon predetermined settings (e.g., when the vehicle 108 exceeds acertain speed or is operating in low-light conditions). In someembodiments, the controller 204 may generate a start signal whencommunication from an external source is received (e.g., when thevehicle 108 is on a smart highway or near another autonomous vehicle).

After receiving the start signal at block 302, the controller 204receives sensor data from the sensors 120 during vehicle operation atblock 304. In some embodiments, the controller 204 may also receiveinformation from external sources through the communication component122 or the communication unit 220. The sensor data may be stored in theRAM 212 for use by the autonomous vehicle operation application 232. Insome embodiments, the sensor data may be recorded in the data storage228 or transmitted to the server 140 via the network 130. The sensordata may alternately either be received by the controller 204 as rawdata measurements from one of the sensors 120 or may be preprocessed bythe sensor 120 prior to being received by the controller 204. Forexample, a tachometer reading may be received as raw data or may bepreprocessed to indicate vehicle movement or position. As anotherexample, a sensor 120 comprising a radar or LIDAR unit may include aprocessor to preprocess the measured signals and send data representingdetected objects in 3-dimensional space to the controller 204.

The autonomous vehicle operation application 232 or other applications230 or routines 240 may cause the controller 204 to process the receivedsensor data at block 306 in accordance with the autonomous operationfeatures. The controller 204 may process the sensor data to determinewhether an autonomous control action is required or to determineadjustments to the controls of the vehicle 108. For example, thecontroller 204 may receive sensor data indicating a decreasing distanceto a nearby object in the vehicle's path and process the received sensordata to determine whether to begin braking (and, if so, how abruptly toslow the vehicle 108). As another example, the controller 204 mayprocess the sensor data to determine whether the vehicle 108 isremaining with its intended path (e.g., within lanes on a roadway). Ifthe vehicle 108 is beginning to drift or slide (e.g., as on ice orwater), the controller 204 may determine appropriate adjustments to thecontrols of the vehicle to maintain the desired bearing. If the vehicle108 is moving within the desired path, the controller 204 maynonetheless determine whether adjustments are required to continuefollowing the desired route (e.g., following a winding road). Under someconditions, the controller 204 may determine to maintain the controlsbased upon the sensor data (e.g., when holding a steady speed on astraight road).

When the controller 204 determines an autonomous control action isrequired at block 308, the controller 204 may cause the controlcomponents of the vehicle 108 to adjust the operating controls of thevehicle to achieve desired operation at block 310. For example, thecontroller 204 may send a signal to open or close the throttle of thevehicle 108 to achieve a desired speed. Alternatively, the controller204 may control the steering of the vehicle 108 to adjust the directionof movement. In some embodiments, the vehicle 108 may transmit a messageor indication of a change in velocity or position using thecommunication component 122 or the communication module 220, whichsignal may be used by other autonomous vehicles to adjust theircontrols. As discussed further below, the controller 204 may also log ortransmit the autonomous control actions to the server 140 via thenetwork 130 for analysis.

The controller 204 may continue to receive and process sensor data atblocks 304 and 306 until an end signal is received by the controller 204at block 312. The end signal may be automatically generated by thecontroller 204 upon the occurrence of certain criteria (e.g., thedestination is reached or environmental conditions require manualoperation of the vehicle 108 by the vehicle operator). Alternatively,the vehicle operator may pause, terminate, or disable the autonomousoperation feature or features using the user-input device or by manuallyoperating the vehicle's controls, such as by depressing a pedal orturning a steering instrument. When the autonomous operation featuresare disabled or terminated, the controller 204 may either continuevehicle operation without the autonomous features or may shut off thevehicle 108, depending upon the circumstances.

Where control of the vehicle 108 must be returned to the vehicleoperator, the controller 204 may alert the vehicle operator in advanceof returning to manual operation. The alert may include a visual, audio,or other indication to obtain the attention of the vehicle operator. Insome embodiments, the controller 204 may further determine whether thevehicle operator is capable of resuming manual operation beforeterminating autonomous operation. If the vehicle operator is determinednot be capable of resuming operation, the controller 204 may cause thevehicle to stop or take other appropriate action.

Exemplary Monitoring Method

FIG. 4A is a flow diagram depicting an exemplary autonomous vehicleoperation monitoring method 400, which may be implemented by theautonomous vehicle insurance system 100. The method 400 monitors theoperation of the vehicle 108 and transmits information regarding thevehicle 108 to the server 140, which information may then be used todetermine autonomous operation feature effectiveness or usage rates toassess risk and price vehicle insurance policy premiums. The method 400may be used both for testing autonomous operation features in acontrolled environment of for determining feature use by an insuredparty. In alternative embodiments, the method 400 may be implementedwhenever the vehicle 108 is in operation (manual or autonomous) or onlywhen the autonomous operation features are enabled. The method 400 maylikewise be implemented as either a real-time process, in whichinformation regarding the vehicle 108 is communicated to the server 140while monitoring is ongoing, or as a periodic process, in which theinformation is stored within the vehicle 108 and communicated to theserver 140 at intervals (e.g., upon completion of a trip or when anincident occurs). In some embodiments, the method 400 may communicatewith the server 140 in real-time when certain conditions exist (e.g.,when a sufficient data connection through the network 130 exists or whenno roaming charges would be incurred).

The method 400 may begin at block 402 when the controller 204 receivesan indication of vehicle operation. The indication may be generated whenthe vehicle 108 is started or when an autonomous operation feature isenabled by the controller 204 or by input from the vehicle operator. Inresponse to receiving the indication, the controller 204 may create atimestamp at block 404. The timestamp may include information regardingthe date, time, location, vehicle environment, vehicle condition, andautonomous operation feature settings or configuration information. Thedate and time may be used to identify one vehicle trip or one period ofautonomous operation feature use, in addition to indicating risk levelsdue to traffic or other factors. The additional location andenvironmental data may include information regarding the position of thevehicle 108 from the GPS unit 206 and its surrounding environment (e.g.,road conditions, weather conditions, nearby traffic conditions, type ofroad, construction conditions, presence of pedestrians, presence ofother obstacles, availability of autonomous communications from externalsources, etc.). Vehicle condition information may include informationregarding the type, make, and model of the vehicle 108, the age ormileage of the vehicle 108, the status of vehicle equipment (e.g., tirepressure, non-functioning lights, fluid levels, etc.), or otherinformation relating to the vehicle 108. In some embodiments, thetimestamp may be recorded on the client device 114, the mobile device110, or the server 140.

The autonomous operation feature settings may correspond to informationregarding the autonomous operation features, such as those describedabove with reference to the autonomous vehicle operation method 300. Theautonomous operation feature configuration information may correspond toinformation regarding the number and type of the sensors 120, thedisposition of the sensors 120 within the vehicle 108, the one or moreautonomous operation features (e.g., the autonomous vehicle operationapplication 232 or the software routines 240), autonomous operationfeature control software, versions of the software applications 230 orroutines 240 implementing the autonomous operation features, or otherrelated information regarding the autonomous operation features. Forexample, the configuration information may include the make and model ofthe vehicle 108 (indicating installed sensors 120 and the type ofon-board computer 114), an indication of a malfunctioning or obscuredsensor 120 in part of the vehicle 108, information regarding additionalafter-market sensors 120 installed within the vehicle 108, a softwareprogram type and version for a control program installed as anapplication 230 on the on-board computer 114, and software program typesand versions for each of a plurality of autonomous operation featuresinstalled as applications 230 or routines 240 in the program memory 208of the on-board computer 114.

During operation, the sensors 120 may generate sensor data regarding thevehicle 108 and its environment. In some embodiments, one or more of thesensors 120 may preprocess the measurements and communicate theresulting processed data to the on-board computer 114. The controller204 may receive sensor data from the sensors 120 at block 406. Thesensor data may include information regarding the vehicle's position,speed, acceleration, direction, and responsiveness to controls. Thesensor data may further include information regarding the location andmovement of obstacles or obstructions (e.g., other vehicles, buildings,barriers, pedestrians, animals, trees, or gates), weather conditions(e.g., precipitation, wind, visibility, or temperature), road conditions(e.g., lane markings, potholes, road material, traction, or slope),signs or signals (e.g., traffic signals, construction signs, buildingsigns or numbers, or control gates), or other information relating tothe vehicle's environment. In some embodiments, sensors 120 may indicatethe number of passengers within the vehicle 108, including an indicationof whether the vehicle is entirely empty.

In addition to receiving sensor data from the sensors 120, in someembodiments the controller 204 may receive autonomous communication datafrom the communication component 122 or the communication module 220 atblock 408. The communication data may include information from otherautonomous vehicles (e.g., sudden changes to vehicle speed or direction,intended vehicle paths, hard braking, vehicle failures, collisions, ormaneuvering or stopping capabilities), infrastructure (road or laneboundaries, bridges, traffic signals, control gates, or emergencystopping areas), or other external sources (e.g., map databases, weatherdatabases, or traffic and accident databases). The communication datamay be combined with the sensor data received at block 406 to obtain amore robust understanding of the vehicle environment. For example, theserver 140 or the controller 204 may combine sensor data indicatingfrequent changes in speed relative to tachometric data with map datarelating to a road upon which the vehicle 108 is traveling to determinethat the vehicle 108 is in an area of hilly terrain. As another example,weather data indicating recent snowfall in the vicinity of the vehicle108 may be combined with sensor data indicating frequent slipping or lowtraction to determine that the vehicle 108 is traveling on asnow-covered or icy road.

At block 410, the controller 204 may process the sensor data, thecommunication data, and the settings or configuration information todetermine whether an incident has occurred. As used herein, an“incident” is an occurrence during operation of an autonomous vehicleoutside of normal safe operating conditions, such that one or more ofthe following occurs: (i) there is an interruption of vehicle operation,(ii) there is damage to the vehicle or other property, (iii) there isinjury to a person, and/or (iv) the conditions require action to betaken by a vehicle operator, autonomous operation feature, pedestrian,or other party to avoid damage or injury. Incidents may includecollisions, hard braking, hard acceleration, evasive maneuvering, lossof traction, detection of objects within a threshold distance from thevehicle 108, alerts presented to the vehicle operator, componentfailure, inconsistent readings from sensors 120, or attemptedunauthorized access to the on-board computer by external sources.Incidents may also include accidents, vehicle breakdowns, flat tires,empty fuel tanks, or medical emergencies. In some embodiments, thecontroller 204 may anticipate or project an expected incident based uponsensor or external data, allowing the controller 204 to send controlsignals to minimize the negative effects of the incident. For example,the controller 204 may cause the vehicle 108 to slow and move to theshoulder of a road immediately before running out of fuel. As anotherexample, adjustable seats within the vehicle 108 may be adjusted tobetter position vehicle occupants in anticipation of a collision.

When an incident is determined to have occurred at block 412,information regarding the incident and the vehicle status may berecorded at block 414, either in the data storage 228 or the database146. The information recorded at block 414 may include sensor data,communication data, and settings or configuration information prior to,during, and immediately following the incident. The information mayfurther include a determination of whether the vehicle 108 has continuedoperating (either autonomously or manually) or whether the vehicle 108is capable of continuing to operate in compliance with applicable safetyand legal requirements. If the controller 204 determines that thevehicle 108 has discontinued operation or is unable to continueoperation at block 416, the method 400 may terminate. If the vehicle 108continues operation, then the method 400 may continue at block 418.

FIG. 4B illustrates an alternative portion of the method 400 followingan incident. When an incident is determined to have occurred at block412, the controller 204 or the server 140 may record status andoperating information at block 414, as above. In some instances, theincident may interrupt communication between the vehicle 108 and theserver 140 via network 130, such that not all information typicallyrecorded will be available for recordation and analysis by the server140. Based upon the data recorded in block 414, the server 140 or thecontroller 204 may determine whether assistance may be needed at thelocation of the vehicle 108 at block 430. For example, the controllermay determine that a head-on collision has occurred based on sensor data(e.g., airbag deployment, automatic motor shut-off, LIDAR dataindicating a collision, etc.) and may further determine based oninformation regarding the speed of the vehicle 108 and other informationthat medical, police, and/or towing services will be necessary. Thedetermination that assistance is needed at block 430 may further includea determination of types of assistance needed (e.g., police, ambulance,fire, towing, vehicle maintenance, fuel delivery, etc.). Thedetermination at block 430 may include analysis of the type of incident,the sensor data regarding the incident (e.g., images from outward facingor inward facing cameras installed within the vehicle, identification ofwhether any passengers were present within the vehicle, determination ofwhether any pedestrians or passengers in other vehicles were involved inthe incident, etc.). The determination of whether assistance is neededat block 430 may further include information regarding the vehiclestatus determined at block 414.

In some embodiments, the determination at block 430 may be supplementedby a verification attempt, such as a phone call or communication throughthe on-board computer 114. Where the verification attempt indicatesassistance is required or communication attempts fail, the server 140 orcontroller 204 would then determine that assistance is needed, asdescribed above. For example, when assistance is determined to be neededat block 430 following an accident involving the vehicle 108, the server140 may direct an automatic telephone call to a mobile telephone numberassociated with the vehicle 108 or the vehicle operator. If no responseis received, or if the respondent indicates assistance is required, theserver 140 may proceed to cause a request for assistance to begenerated.

When assistance is determined to be needed at block 432, the controller204 or the server 140 may send a request for assistance at block 434.The request may include information regarding the vehicle 108, such asthe vehicle's location, the type of assistance required, other vehiclesinvolved in the incident, the pedestrians involved in the incident,vehicle operators or passengers involved in the incident, and/or otherrelevant information. The request for assistance may include telephonic,data, or other requests to one or more emergency or vehicular serviceproviders (e.g., local police, fire departments, state highway patrols,emergency medical services, public or private ambulance services,hospitals, towing companies, roadside assistance services, vehiclerental services, local claims representative offices, etc.). Aftersending a request for assistance at block 434 or when assistance isdetermined not to be needed at block 432, the controller 204 or theserver 140 may next determine whether the vehicle is operational atblock 416, as described above. The method 400 may then end or continueas indicated in FIG. 4A.

In some embodiments, the controller 204 may further determineinformation regarding the likely cause of a collision or other incident.Alternatively, or additionally, the server 140 may receive informationregarding an incident from the on-board computer 114 and determinerelevant additional information regarding the incident from the sensordata. For example, the sensor data may be used to determine the pointsof impact on the vehicle 108 and another vehicle involved in acollision, the relative velocities of each vehicle, the road conditionsat the time of the incident, and the likely cause or the party likely atfault. This information may be used to determine risk levels associatedwith autonomous vehicle operation, as described below, even where theincident is not reported to the insurer.

At block 418, the controller 204 may determine whether a change oradjustment to one or more of the settings or configuration of theautonomous operation features has occurred. Changes to the settings mayinclude enabling or disabling an autonomous operation feature oradjusting the feature's parameters (e.g., resetting the speed on anadaptive cruise control feature). If the settings or configuration aredetermined to have changed, the new settings or configuration may berecorded at block 422, either in the data storage 228 or the database146.

At block 424, the controller 204 may record the operating data relatingto the vehicle 108 in the data storage 228 or communicate the operatingdata to the server 140 via the network 130 for recordation in thedatabase 146. The operating data may include the settings orconfiguration information, the sensor data, and the communication datadiscussed above, as well as data regarding control decisions generatedby one or more autonomous operation features, as discussed below. Insome embodiments, operating data related to normal autonomous operationof the vehicle 108 may be recorded. In other embodiments, only operatingdata related to incidents of interest may be recorded, and operatingdata related to normal operation may not be recorded. In still otherembodiments, operating data may be stored in the data storage 228 untila sufficient connection to the network 130 is established, but some orall types of incident information may be transmitted to the server 140using any available connection via the network 130.

At block 426, the controller 204 may determine whether the vehicle 108is continuing to operate. In some embodiments, the method 400 mayterminate when all autonomous operation features are disabled, in whichcase the controller 204 may determine whether any autonomous operationfeatures remain enabled at block 426. When the vehicle 108 is determinedto be operating (or operating with at least one autonomous operationfeature enabled) at block 426, the method 400 may continue throughblocks 406-426 until vehicle operation has ended. When the vehicle 108is determined to have ceased operating (or is operating withoutautonomous operation features enabled) at block 426, the controller 204may record the completion of operation at block 428, either in the datastorage 228 or the database 146. In some embodiments, a second timestampcorresponding to the completion of vehicle operation may likewise berecorded, as above.

Exemplary Evaluation Methods

FIG. 5 illustrates a flow diagram of an exemplary autonomous operationfeature evaluation method 500 for determining the effectiveness ofautonomous operation features, which may be implemented by theautonomous vehicle insurance system 100. The method 500 begins bymonitoring and recording the responses of an autonomous operationfeature in a test environment at block 502. The test results are thenused to determine a plurality of risk levels for the autonomousoperation feature corresponding to the effectiveness of the feature insituations involving various conditions, configurations, and settings atblock 504. Once a baseline risk profile of the plurality of risk levelshas been established at block 504, the method 500 may refine or adjustthe risk levels based upon operating data and actual losses for insuredautonomous vehicles operation outside the test environment in blocks506-510. Although FIG. 5 shows the method for only one autonomousoperation feature, it should be understood that the method 500 may beperformed to evaluate each of any number of autonomous operationfeatures or combinations of autonomous operation features. In someembodiments, the method 500 may be implemented for a plurality ofautonomous operation features concurrently on multiple servers 140 or atdifferent times on one or more servers 140.

At block 502, the effectiveness of an autonomous operation feature istested in a controlled testing environment by presenting test conditionsand recording the responses of the feature. The testing environment mayinclude a physical environment in which the autonomous operation featureis tested in one or more vehicles 108. Additionally, or alternatively,the testing environment may include a virtual environment implemented onthe server 140 or another computer system in which the responses of theautonomous operation feature are simulated. Physical or virtual testingmay be performed for a plurality of vehicles 108 and sensors 120 orsensor configurations, as well as for multiple settings of theautonomous operation feature. In some embodiments, the compatibility orincompatibility of the autonomous operation feature with vehicles 108,sensors 120, communication units 122, on-board computers 114, controlsoftware, or other autonomous operation features may be tested byobserving and recording the results of a plurality of combinations ofthese with the autonomous operation feature. For example, an autonomousoperation feature may perform well in congested city traffic conditions,but that will be of little use if it is installed in an automobile withcontrol software that operates only above 30 miles per hour.Additionally, some embodiments may further test the response ofautonomous operation features or control software to attempts atunauthorized access (e.g., computer hacking attempts), which results maybe used to determine the stability or reliability of the autonomousoperation feature or control software.

The test results may be recorded by the server 140. The test results mayinclude responses of the autonomous operation feature to the testconditions, along with configuration and setting data, which may bereceived by the on-board computer 114 and communicated to the server140. During testing, the on-board computer 114 may be a special-purposecomputer or a general-purpose computer configured for generating orreceiving information relating to the responses of the autonomousoperation feature to test scenarios. In some embodiments, additionalsensors may be installed within the vehicle 108 or in the vehicleenvironment to provide additional information regarding the response ofthe autonomous operation feature to the test conditions, whichadditional sensors may not provide sensor data to the autonomousoperation feature.

In some embodiments, new versions of previously tested autonomousoperation features may not be separately tested, in which case the block502 may not be present in the method 500. In such embodiments, theserver 140 may determine the risk levels associated with the new versionby reference to the risk profile of the previous version of theautonomous operation feature in block 504, which may be adjusted basedupon actual losses and operating data in blocks 506-510. In otherembodiments, each version of the autonomous operation feature may beseparately tested, either physically or virtually. Alternatively, oradditionally, a limited test of the new version of the autonomousoperation feature may be performed and compared to the test results ofthe previous version, such that additional testing may not be performedwhen the limited test results of the new version are within apredetermined range based upon the test results of the previous version.

FIG. 6 illustrates a flow diagram of an exemplary autonomous operationfeature testing method 600 for presenting test conditions to anautonomous operation feature and observing and recording responses tothe test conditions in accordance with the method 500. Although themethod 600 is illustrated for one autonomous operation feature, itshould be understood that the exemplary method 600 may be performed totest any number of features or combinations of features. At block 602,the server 140 may determine the scope of the testing based upon theautonomous operation feature and the availability of test results forrelated or similar autonomous operation features (e.g., previousversions of the feature). The scope of the testing may includeparameters such as configurations, settings, vehicles 108, sensors 120,communication units 122, on-board computers 114, control software, otherautonomous operation features, or combinations of these parameters to betested.

At block 604, the autonomous operation feature is enabled within a testsystem with a set of parameters determined in block 602. The test systemmay be a vehicle 108 or a computer simulation, as discussed above. Theautonomous operation feature or the test system may be configured toprovide the desired parameter inputs to the autonomous operationfeature. For example, the controller 204 may disable a number of sensors120 or may provide only a subset of available sensor data to theautonomous operation feature for the purpose of testing the feature'sresponse to certain parameters.

At block 606, test inputs are presented to the autonomous operationfeature, and responses of the autonomous operation feature are observedat block 608. The test inputs may include simulated data presented bythe on-board computer 114 or sensor data from the sensors 120 within thevehicle 108. In some embodiments, the vehicle 108 may be controlledwithin a physical test environment by the on-board computer 114 topresent desired test inputs through the sensors 120. For example, theon-board computer 114 may control the vehicle 108 to maneuver nearobstructions or obstacles, accelerate, or change directions to triggerresponses from the autonomous operation feature. The test inputs mayalso include variations in the environmental conditions of the vehicle108, such as by simulating weather conditions that may affect theperformance of the autonomous operation feature (e.g., snow or ice coveron a roadway, rain, or gusting crosswinds, etc.).

In some embodiments, additional vehicles may be used to test theresponses of the autonomous operation feature to moving obstacles. Theseadditional vehicles may likewise be controlled by on-board computers orremotely by the server 140 through the network 130. In some embodiments,the additional vehicles may transmit autonomous communicationinformation to the vehicle 108, which may be received by thecommunication component 122 or the communication unit 220 and presentedto the autonomous operation feature by the on-board computer 114. Thus,the response of the autonomous operation feature may be tested with andwithout autonomous communications from external sources. The responsesof the autonomous operation feature may be observed as output signalsfrom the autonomous operation feature to the on-board computer 114 orthe vehicle controls. Additionally, or alternatively, the responses maybe observed by sensor data from the sensors 120 and additional sensorswithin the vehicle 108 or placed within the vehicle environment.

At block 610, the observed responses of the autonomous operation featureare recorded for use in determining effectiveness of the feature. Theresponses may be recorded in the data storage 228 of the on-boardcomputer 114 or in the database 146 of the server 140. If the responsesare stored on the on-board computer 114 during testing, the results maybe communicated to the server 140 via the network either during or aftercompletion of testing.

At block 612, the on-board computer 114 or the server 140 may determinewhether the additional sets of parameters remain for which theautonomous operation feature is to be tested, as determined in block602. When additional parameter sets are determined to remain at block612, they are separately tested according to blocks 604-610. When noadditional parameter sets are determined to exist at block 612, themethod 600 terminates.

Although the method 600 is discussed above as testing the autonomous(and/or semi-autonomous) operation features in a test vehicle operatingwithin a test environment, it should be understood that the exemplarymethod 600 may be similarly performed in an uncontrolled environment(i.e., on public roadways) or in a virtual environment. Testing ofautonomous features within a virtual environment may include thepresentation of electrical signals mimicking signals generated by one ormore sensors in a plurality of operating scenarios at block 606. Forexample, a control unit or on-board computer 114 removed from a vehiclemay be connected to one or more sensor input simulators (e.g., acomputer or computer-controlled signal generator) that present inputsignals to the control unit or on-board computer that correspond tosignals that would be received from the sensors 120 in the vehicle 108under certain conditions. In such case, the same or another computer maybe connected to the control unit or on-board computer 114 to receive andrecord the control outputs determined by the one or more autonomousoperation features in response to the simulated sensor input at blocks608 and 610.

Additionally, or alternatively, the virtual test environment may includea simulation of an autonomous (and/or semi-autonomous) operation featurerunning on a general-purpose or special-purpose computer system. In suchan embodiment, the autonomous operation feature may include one or moresoftware routines, processes, applications, or modules that implementthe autonomous operation feature to generate control signals for avehicle when executed on a general-purpose or special-purpose processor.For example, an adaptive cruise control feature may include a softwareroutine to monitor the speed of the vehicle using a combination ofspeedometer and other sensor data, as well as a software routine todetermine the distance of the vehicle from obstacles or other vehiclesin the vehicle's path using LIDAR and autonomous communication data. Theadaptive cruise control feature may further include a control softwareroutine to determine adjustments to the vehicle's speed to maintain asafe distance from other vehicles and obstacles then generate a controlsignal to maintain or adjust the throttle of the vehicle. In a virtualtest environment, the software routines of the autonomous operationfeature may be executed on a processor of a computer system notconnected to any vehicle, in which case test input signals simulatingsignals from sensors within a vehicle may be presented to the softwareroutines to test the routines' responses. Thus, a process of thecomputer system may execute instructions causing the processor to accessa set of simulated sensor test input signals at block 606, determine aresponse (such as one or more output test signals) of the autonomousoperation feature based on the executable instructions representing oneor more software routines of the autonomous operation feature at block608, and record the determined response at block 610.

As the software routines of the autonomous operation feature are notdirectly connected to a vehicle to control the vehicle's operationduring virtual testing, virtual testing may further include predicting aresponse of a vehicle to output test signals generated by the softwareroutines of the tested autonomous operation feature. The simulatedsensor inputs and/or test input signals may include sets of datacorresponding to generated or recorded signals from a plurality ofsensors. In some embodiments, the computer system may access a first setof sensor data and sequentially update the sensor data using thedetermined responses of the simulated autonomous operation feature.Thus, the test input signals may include input signals stored in adatabase and accessed by a testing program or application forpresentation to the software routines of the autonomous operationfeature being tested. Additionally, or alternatively, the computersystem may generate sensor input signals based on a simulation of aphysical test environment and update the virtual test environment basedon the determined responses of the autonomous operation features. Insome embodiments, a virtual testing program or application may furthercontrol the virtual testing process and may simulate the operation of avirtual vehicle within a virtual test environment. The virtual testenvironment may include a computer simulation of an environment in whichan autonomous vehicle may be expected to operate (e.g., a congestedhighway, a city street with multiple intersections, etc.). The virtualtesting program or application may, thus, generate a simulation of theoperation of the virtual vehicle in the virtual test environment,including vehicle attributes such as position, speed, or momentum. Thevirtual testing program may further simulate the presence or operationof other vehicles, traffic control devices, obstacles, pedestrians, orother relevant features of a vehicle environment. Based upon thesesimulated features of the virtual vehicle environment, one or moresimulated sensor readings or data may be determined, which may furtherbe used to determine one or more test input signals to the softwareroutines of the autonomous operation feature. The virtual testingprogram or application may then present the test input signals to thesoftware routines and cause test output signals to be generated inresponse to the test input signals. From these test output signals, thevirtual testing program may then predict the response of the virtualvehicle to the test output signals, including the responses of thevirtual vehicle in relation to other virtual vehicles or other featuresin the simulated virtual test environment.

In any of the foregoing virtual test environments, the input data mayinclude sensor data recorded during operation of an autonomous (and/orsemi-autonomous) vehicle 108, which may include operation by a vehicleoperator or by other autonomous (and/or semi-autonomous) operationfeatures. For example, a vehicle operator may choose to operate thevehicle 108 manually under some conditions (e.g., snow, fog, orconstruction), or the autonomous operation features may not supportautonomous operation under such conditions. The sensors 120 of thevehicle 108 may, however, continue to collect and record data regardingthe surrounding environment. The sensor data may then be used tosimulate autonomous operation feature responses (i.e., the controlsignals the autonomous operation feature would have generated had itbeen in control of the vehicle). The data and responses may be stored ortransmitted via the network 130, and the responses of the autonomousoperation features may be determined at the time of operation or at alater time.

As discussed with reference to FIG. 5 and elsewhere herein, theeffectiveness of one or more autonomous operation features may befurther used to determine one or more risk levels or risk profilesassociated with the autonomous operation features. Specifically, thetest output responses generated by the software routines of theautonomous operation features or the predicted responses of the virtualvehicle to the test output responses may be compared with other similarvirtual test data related to other autonomous operation features. Whereavailable, actual observed operating data regarding the one or moreother autonomous operation features disposed within a plurality of othervehicles operating outside the virtual test environment may also becompared with the virtual test data using known statistical modelingtechniques. Additionally, actual loss data for vehicles operatingoutside the virtual test environment and having the other autonomousoperating features may further be compared with the virtual test data tobetter assess the risk levels and risk profiles associated with thetested autonomous operating features.

It will be apparent that performance of the exemplary method 600 in avirtual test environment offers advantages in terms of cost and time.Once set up, hundreds or thousands of test scenarios may beautomatically run to evaluate autonomous (and/or semi-autonomous)operation feature performance under a variety of conditions withoutinput from a user or vehicle operator. For example, a new autonomousoperation feature or a software update including a new version of anautonomous operation feature may be tested in a virtual test environmentprior to installation within autonomous vehicles, allowing immediateadjustment of risk levels or risk profiles for vehicles using the newautonomous operation feature or version. In this way, adjustments torisks associated with autonomous operation features may be made withoutreference to actual loss data relating to the specific autonomousoperation features. Such advantages must be weighed against thelimitations of virtual testing, however, because the test results arelimited by the quality of the virtual test environment. It will bereadily apparent that responses from physical and virtual testenvironments may be combined in order to determine the performance andrisk levels associated with autonomous operation features.

Referring again to FIG. 5, the server 140 may determine a baseline riskprofile for the autonomous operation feature from the recorded testresults at block 504, including a plurality of risk levels correspondingto a plurality of sets of parameters such as configurations, settings,vehicles 108, sensors 120, communication units 122, on-board computers114, control software, other autonomous operation features, orcombinations of these. The server 140 may determine the risk levelsassociated with the autonomous operation feature by implementing thefeature evaluation application 142 to determine the effectiveness of thefeature. In some embodiments, the server 140 may further implement thecompatibility evaluation application 143 to determine the effectivenessof combinations of features based upon test results and otherinformation. Additionally, or alternatively, in some embodiments, thebaseline risk profile may not depend upon the type, make, model, year,or other aspect of the vehicle 108. In such embodiments, the baselinerisk profile and adjusted risk profiles may correspond to theeffectiveness or risk levels associated with the autonomous operationfeatures across a range of vehicles, disregarding any variations ineffectiveness or risk levels associated with operation of the featuresin different vehicles.

FIG. 7 illustrates a flow diagram of an exemplary autonomous featureevaluation method 700 for determining the effectiveness of an autonomousoperation feature under a set of environmental conditions, configurationconditions, and settings. Although the method 700 shows determination ofa risk level associated with an autonomous operation feature within oneset of parameters, it should be understood that the method 700 may beimplemented for any number of sets of parameters for any number ofautonomous features or combinations thereof.

At block 702, the server 140 receives the test result data observed andrecorded in block 502 for the autonomous operation feature inconjunction with a set of parameters. In some embodiments, the restresult data may be received from the on-board computer 114 or from thedatabase 146. In addition, in some embodiments, the server 140 mayreceive reference data for other autonomous operation features in use oninsured autonomous vehicles at block 704, such as test result data andcorresponding actual loss or operating data for the other autonomousoperation features. The reference data received at block 704 may belimited to data for other autonomous operation features havingsufficient similarity to the autonomous operation feature beingevaluated, such as those performing a similar function, those withsimilar test result data, or those meeting a minimum threshold level ofactual loss or operating data.

Using the test result data received at block 702 and the reference datareceived at block 704, the server 140 determines the expected actualloss or operating data for the autonomous operation feature at block706. The server 140 may determine the expected actual loss or operatingdata using known techniques, such as regression analysis or machinelearning tools (e.g., neural network algorithms or support vectormachines). The expected actual loss or operating data may be determinedusing any useful metrics, such as expected loss value, expectedprobabilities of a plurality of collisions or other incidents, expectedcollisions per unit time or distance traveled by the vehicle, etc.

At block 708, the server 140 may further determine a risk levelassociated with the autonomous operation feature in conjunction with theset of parameters received in block 702. The risk level may be a metricindicating the risk of collision, malfunction, or other incident leadingto a loss or claim against a vehicle insurance policy covering a vehiclein which the autonomous operation feature is functioning. The risk levelmay be defined in various alternative ways, including as a probabilityof loss per unit time or distance traveled, a percentage of collisionsavoided, or a score on a fixed scale. In a preferred embodiment, therisk level is defined as an effectiveness rating score such that ahigher score corresponds to a lower risk of loss associated with theautonomous operation feature.

Referring again to FIG. 5, the method 700 may be implemented for eachrelevant combination of an autonomous operation feature in conjunctionwith a set of parameters relating to environmental conditions,configuration conditions, and settings. It may be beneficial in someembodiments to align the expected losses or operating data metrics withloss categories for vehicle insurance policies. Once the baseline riskprofile is determined for the autonomous operation feature, theplurality of risk levels in the risk profile may be updated or adjustedin blocks 506-510 using actual loss and operating data from autonomousvehicles operating in the ordinary course, viz. not in a testenvironment.

At block 506, the server 140 may receive operating data from one or morevehicles 108 via the network 130 regarding operation of the autonomousoperation feature. The operating data may include the operating datadiscussed above with respect to monitoring method 400, includinginformation regarding the vehicle 108, the vehicle's environment, thesensors 120, communications for external sources, the type and versionof the autonomous operation feature, the operation of the feature, theconfiguration and settings relating to the operation of the feature, theoperation of other autonomous operation features, control actionsperformed by the vehicle operator, or the location and time ofoperation. The operating data may be received by the server 140 from theon-board computer 114 or the mobile device 110 implementing themonitoring method 400 or from other sources, and the server 140 mayreceive the operating data either periodically or continually.

At block 508, the server 140 may receive data regarding actual losses onautonomous vehicles that included the autonomous operation feature. Thisinformation may include claims filed pursuant to insurance policies,claims paid pursuant to insurance policies, accident reports filed withgovernment agencies, or data from the sensors 120 regarding incidents(e.g., collisions, alerts presented, etc.). This actual loss informationmay further include details such as date, time, location, trafficconditions, weather conditions, road conditions, vehicle speed, vehicleheading, vehicle operating status, autonomous operation featureconfiguration and settings, autonomous communications transmitted orreceived, points of contact in a collision, velocity and movements ofother vehicles, or additional information relevant to determining thecircumstances involved in the actual loss.

At block 510, the server 140 may process the information received atblocks 506 and 508 to determine adjustments to the risk levelsdetermined at block 504 based upon actual loss and operating data forthe autonomous operation feature. Adjustments may be necessary becauseof factors such as sensor failure, interference disrupting autonomouscommunication, better or worse than expected performance in heavytraffic conditions, etc. The adjustments to the risk levels may be madeby methods similar to those used to determine the baseline risk profilefor the autonomous operation feature or by other known methods (e.g.,Bayesian updating algorithms). The updating procedure of blocks 506-510may be repeatedly implemented periodically or continually as new databecome available to refine and update the risk levels or risk profileassociated with the autonomous operation feature. In subsequentiterations, the most recently updated risk profile or risk levels may beadjusted, rather than the initial baseline risk profile or risk levelsdetermined in block 504.

Exemplary Autonomous Vehicle Insurance Risk and Price DeterminationMethods

The risk profiles or risk levels associated with one or more autonomousoperation features determined above may be further used to determinerisk categories or premiums for vehicle insurance policies coveringautonomous vehicles. FIGS. 8-10 illustrate flow diagrams of exemplaryembodiments of methods for determining risk associated with anautonomous vehicle or premiums for vehicle insurance policies coveringan autonomous vehicle. In some embodiments or under some conditions, theautonomous vehicle may be a fully autonomous vehicle operating without avehicle operator's input or presence. In other embodiments or underother conditions, the vehicle operator may control the vehicle with orwithout the assistance of the vehicle's autonomous operation features.For example, the vehicle may be fully autonomous only above a minimumspeed threshold or may require the vehicle operator to control thevehicle during periods of heavy precipitation. Alternatively, theautonomous vehicle may perform all relevant control functions using theautonomous operation features under all ordinary operating conditions.In still further embodiments, the vehicle 108 may operate in either afully or a partially autonomous state, while receiving or transmittingautonomous communications.

Where the vehicle 108 operates only under fully autonomous control bythe autonomous operation features under ordinary operating conditions orwhere control by a vehicle operator may be disregarded for insurancerisk and price determination, the method 800 may be implemented todetermine the risk level or premium associated with an insurance policycovering the autonomous vehicle. Where the vehicle 108 may be operatedmanually under some conditions, the method 900 may be implemented todetermine the risk level or premium associated with an insurance policycovering the autonomous vehicle, including a determination of the risksassociated with the vehicle operator performing manual vehicleoperation. Where the vehicle 108 may be operated with the assistance ofautonomous communications features, the method 1000 may be implementedto determine the risk level or premium associated with an insurancepolicy covering the autonomous vehicle, including a determination of theexpected use of autonomous communication features by external sources inthe relevant environment of the vehicle 108 during operation of thevehicle 108.

FIG. 8 illustrates a flow diagram depicting an exemplary embodiment of afully autonomous vehicle insurance pricing method 800, which may beimplemented by the autonomous vehicle insurance system 100. The method800 may be implemented by the server 140 to determine a risk level orprice for a vehicle insurance policy covering a fully autonomous vehiclebased upon the risk profiles of the autonomous operation features in thevehicle. It is important to note that the risk category or price isdetermined without reference to factors relating to risks associatedwith a vehicle operator (e.g., age, experience, prior history of vehicleoperation). Instead, the risk and price may be determined based upon thevehicle 108, the location and use of the vehicle 108, and the autonomousoperation features of the vehicle 108.

At block 802, the server 140 receives a request to determine a riskcategory or premium associated with a vehicle insurance policy for afully autonomous vehicle. The request may be caused by a vehicleoperator or other customer or potential customer of an insurer, or by aninsurance broker or agent. The request may also be generatedautomatically (e.g., periodically for repricing or renewal of anexisting vehicle insurance policy). In some instances, the server 140may generate the request upon the occurrence of specified conditions.

At block 804, the server 140 receives information regarding the vehicle108, the autonomous operation features installed within the vehicle 108,and anticipated or past use of the vehicle 108. The information mayinclude vehicle information (e.g., type, make, model, year ofproduction, safety features, modifications, installed sensors, on-boardcomputer information, etc.), autonomous operation features (e.g., type,version, connected sensors, compatibility information, etc.), and useinformation (e.g., primary storage location, primary use, primaryoperating time, past use as monitored by an on-board computer or mobiledevice, past use of one or more vehicle operators of other vehicles,etc.). The information may be provided by a person having an interest inthe vehicle, a customer, or a vehicle operator, and/or the informationmay be provided in response to a request for the information by theserver 140. Alternatively, or additionally, the server 140 may requestor receive the information from one or more databases communicativelyconnected to the server 140 through the network 130, which may includedatabases maintained by third parties (e.g., vehicle manufacturers orautonomous operation feature manufacturers). In some embodiments,information regarding the vehicle 108 may be excluded, in which case therisk or premium determinations below may likewise exclude theinformation regarding the vehicle 108.

At block 806, the server 140 may determine the risk profile or risklevels associated with the vehicle 108 based upon the vehicleinformation and the autonomous operation feature information received atblock 804. The risk levels associated with the vehicle 108 may bedetermined as discussed above with respect to the method 500 and/or maybe determined by looking up in a database the risk level informationpreviously determined. In some embodiments, the information regardingthe vehicle may be given little or no weight in determining the risklevels. In other embodiments, the risk levels may be determined basedupon a combination of the vehicle information and the autonomousoperation information. As with the risk levels associated with theautonomous operation features discussed above, the risk levelsassociated with the vehicle may correspond to the expected losses orincidents for the vehicle based upon its autonomous operation features,configuration, settings, and/or environmental conditions of operation.For example, a vehicle may have a risk level of 98% effectiveness whenon highways during fair weather days and a risk level of 87%effectiveness when operating on city streets at night in moderate rain.A plurality of risk levels associated with the vehicle may be combinedwith estimates of anticipated vehicle use conditions to determine thetotal risk associated with the vehicle.

At block 808, the server 140 may determine the expected use of thevehicle 108 in the relevant conditions or with the relevant settings tofacilitate determining a total risk for the vehicle 108. The server 140may determine expected vehicle use based upon the use informationreceived at block 804, which may include a history of prior use recordedby the vehicle 108 and/or another vehicle. For example, recorded vehicleuse information may indicate that 80% of vehicle use occurs duringweekday rush hours in or near a large city, that 20% occurs on nightsand weekends. From this information, the server 140 may determine that80% (75%, 90%, etc.) of the expected use of the vehicle 108 is in heavytraffic and that 20% (25%, 10%, etc.) is in light traffic. The server140 may further determine that vehicle use is expected to be 60% onlimited access highways and 40% on surface streets. Based upon thevehicle's typical storage location, the server 140 may access weatherdata for the location to determine expected weather conditions duringthe relevant times. For example, the server 140 may determine that 20%of the vehicle's operation on surface streets in heavy traffic willoccur in rain or snow. In a similar manner, the server 140 may determinea plurality of sets of expected vehicle use parameters corresponding tothe conditions of use of the vehicle 108. These conditions may furthercorrespond to situations in which different autonomous operationfeatures may be engaged and/or may be controlling the vehicle.Additionally, or alternatively, the vehicle use parameters maycorrespond to different risk levels associated with the autonomousoperation features. In some embodiments, the expected vehicle useparameters may be matched to the most relevant vehicle risk levelparameters, viz. the parameters corresponding to vehicle risk levelsthat have the greatest predictive effect and/or explanatory power.

At block 810, the server 140 may use the risk levels determined at block806 and the expected vehicle use levels determined at block 808 todetermine a total expected risk level. To this end, it may beadvantageous to attempt to match the vehicle use parameters as closelyas possible to the vehicle risk level parameters. For example, theserver 140 may determine the risk level associated with each of aplurality of sets of expected vehicle use parameters. In someembodiments, sets of vehicle use parameters corresponding to zero ornegligible (e.g., below a predetermined threshold probability) expecteduse levels may be excluded from the determination for computationalefficiency. The server 140 may then weight the risk levels by thecorresponding expected vehicle use levels, and aggregate the weightedrisk levels to obtain a total risk level for the vehicle 108. In someembodiments, the aggregated weighted risk levels may be adjusted ornormalized to obtain the total risk level for the vehicle 108. In someembodiments, the total risk level may correspond to a regulatory riskcategory or class of a relevant insurance regulator.

At block 812, the server 140 may determine one or more premiums forvehicle insurance policies covering the vehicle 108 based upon the totalrisk level determined at block 810. These policy premiums may also bedetermine based upon additional factors, such as coverage type and/oramount, expected cost to repair or replace the vehicle 108, expectedcost per claim for liability in the locations where the vehicle 108 istypically used, discounts for other insurance coverage with the sameinsurer, and/or other factors unrelated to the vehicle operator. In someembodiments, the server 140 may further communicate the one or morepolicy premiums to a customer, broker, agent, or other requesting personor organization via the network 130. The server 140 may further storethe one or more premiums in the database 146.

FIG. 9 illustrates a flow diagram depicting an exemplary embodiment of apartially autonomous vehicle insurance pricing method 900, which may beimplemented by the autonomous vehicle insurance system 100 in a mannersimilar to that of the method 800. The method 900 may be implemented bythe server 140 to determine a risk category and/or price for a vehicleinsurance policy covering an autonomous vehicle based upon the riskprofiles of the autonomous operation features in the vehicle and/or theexpected use of the autonomous operation features. In addition toinformation regarding the vehicle 108 and the autonomous operationfeatures, the method 900 includes information regarding the vehicleoperator, including information regarding the expected use of theautonomous operation features and/or the expected settings of thefeatures under various conditions. Such additional information isrelevant where the vehicle operator may control the vehicle 108 undersome conditions and/or may determine settings affecting theeffectiveness of the autonomous operation features.

At block 902, the server 140 may receive a request to determine a riskcategory and/or premium associated with a vehicle insurance policy foran autonomous vehicle in a manner similar to block 802 described above.At block 904, the server 140 likewise receives information regarding thevehicle 108, the autonomous operation features installed within thevehicle 108, and/or anticipated or past use of the vehicle 108. Theinformation regarding anticipated or past use of the vehicle 108 mayinclude information regarding past use of one or more autonomousoperation features, and/or settings associated with use of the features.For example, this may include times, road conditions, and/or weatherconditions when autonomous operation features have been used, as well assimilar information for past vehicle operation when the features havebeen disabled. In some embodiments, information regarding the vehicle108 may be excluded, in which case the risk or premium determinationsbelow may likewise exclude the information regarding the vehicle 108. Atblock 906, the server 140 may receive information related to the vehicleoperator, including standard information of a type typically used inactuarial analysis of vehicle operator risk (e.g., age, location, yearsof vehicle operation experience, and/or vehicle operating history of thevehicle operator).

At block 908, the server 140 may determine the risk profile or risklevels associated with the vehicle 108 based upon the vehicleinformation and the autonomous operation feature information received atblock 904. The risk levels associated with the vehicle 108 may bedetermined as discussed above with respect to the method 500 and/or asfurther discussed with respect to method 800.

At block 910, the server 140 may determine the expected manual and/orautonomous use of the vehicle 108 in the relevant conditions and/or withthe relevant settings to facilitate determining a total risk for thevehicle 108. The server 140 may determine expected vehicle use basedupon the use information received at block 904, which may include ahistory of prior use recorded by the vehicle 108 and/or another vehiclefor the vehicle operator. Expected manual and autonomous use of thevehicle 108 may be determined in a manner similar to that discussedabove with respect to method 800, but including an additionaldetermination of the likelihood of autonomous and/or manual operation bythe vehicle operation under the various conditions. For example, theserver 140 may determine based upon past operating data that the vehicleoperator manually controls the vehicle 108 when on a limited-accesshighway only 20% of the time in all relevant environments, but the samevehicle operator controls the vehicle 60% of the time on surface streetsoutside of weekday rush hours and 35% of the time on surface streetsduring weekday rush hours. These determinations may be used to furtherdetermine the total risk associated with both manual and/or autonomousvehicle operation.

At block 912, the server 140 may use the risk levels determined at block908 and the expected vehicle use levels determined at block 910 todetermine a total expected risk level, including both manual andautonomous operation of the vehicle 108. The autonomous operation risklevels may be determined as above with respect to block 810. The manualoperation risk levels may be determined in a similar manner, but themanual operation risk may include risk factors related to the vehicleoperator. In some embodiments, the manual operation risk may also bedetermined based upon vehicle use parameters and/or related autonomousoperation feature risk levels for features that assist the vehicleoperator in safely controlling the vehicle. Such features may includealerts, warnings, automatic braking for collision avoidance, and/orsimilar features that may provide information to the vehicle operator ortake control of the vehicle from the vehicle operator under someconditions. These autonomous operation features may likewise beassociated with different risk levels that depend upon settings selectedby the vehicle operator. Once the risk levels associated with autonomousoperation and manual operation under various parameter sets that havebeen weighted by the expected use levels, the total risk level for thevehicle and operator may be determined by aggregating the weighted risklevels. As above, the total risk level may be adjusted or normalized,and/or it may be used to determine a risk category or risk class inaccordance with regulatory requirements.

At block 914, the server 140 may determine one or more premiums forvehicle insurance policies covering the vehicle 108 based upon the totalrisk level determined at block 812. As in method 800, additional factorsmay be included in the determination of the policy premiums, and/or thepremiums may be adjusted based upon additional factors. The server 140may further record the premiums or may transmit one or more of thepolicy premiums to relevant parties.

FIG. 10 illustrates a flow diagram depicting an exemplary embodiment ofan autonomous vehicle insurance pricing method 1000 for determining riskand/or premiums for vehicle insurance policies covering autonomousvehicles with autonomous communication features, which may beimplemented by the autonomous vehicle insurance system 100. The method1000 may determine risk levels as without autonomous communicationdiscussed above with reference to methods 800 and/or 900, then adjustthe risk levels based upon the availability and effectiveness ofcommunications between the vehicle 108 and external sources. Similar toenvironmental conditions, the availability of external sources such asother autonomous vehicles for communication with the vehicle 108 affectsthe risk levels associated with the vehicle 108. For example, use of anautonomous communication feature may significantly reduce riskassociated with autonomous operation of the vehicle 108 only where otherautonomous vehicles also use autonomous communication features to sendand/or receive information.

At block 1002, the server 140 may receive a request to determine a riskcategory or premium associated with a vehicle insurance policy for anautonomous vehicle with one or more autonomous communication features ina manner similar to blocks 802 and/or 902 described above. At block1004, the server 140 likewise receives information regarding the vehicle108, the autonomous operation features installed within the vehicle 108(including autonomous communication features), the vehicle operator,and/or anticipated or past use of the vehicle 108. The informationregarding anticipated or past use of the vehicle 108 may includeinformation regarding locations and times of past use, as well as pastuse of one or more autonomous communication features. For example, thismay include locations, times, and/or details of communication exchangedby an autonomous communication feature, as well as information regardingpast vehicle operation when no autonomous communication occurred. Thisinformation may be used to determine the past availability of externalsources for autonomous communication with the vehicle 108, facilitatingdetermination of expected future availability of autonomouscommunication as described below. In some embodiments, informationregarding the vehicle 108 may be excluded, in which case the risk orpremium determinations below may likewise exclude the informationregarding the vehicle 108.

At block 1006, the server 140 may determine the risk profile or risklevels associated with the vehicle 108 based upon the vehicleinformation, the autonomous operation feature information, and/or thevehicle operator information received at block 1004. The risk levelsassociated with the vehicle 108 may be determined as discussed abovewith respect to the method 500 and as further discussed with respect tomethods 800 and 900. At block 1008, the server 140 may determine therisk profile and/or risk levels associated with the vehicle 108 and/orthe autonomous communication features. This may include a plurality ofrisk levels associated with a plurality of autonomous communicationlevels and/or other parameters relating to the vehicle 108, the vehicleoperator, the autonomous operation features, the configuration and/orsetting of the autonomous operation features, and/or the vehicle'senvironment. The autonomous communication levels may include informationregarding the proportion of vehicles in the vehicle's environment thatare in autonomous communication with the vehicle 108, levels ofcommunication with infrastructure, types of communication (e.g., hardbraking alerts, full velocity information, etc.), and/or otherinformation relating to the frequency and/or quality of autonomouscommunications between the autonomous communication feature and theexternal sources.

At block 1010, the server 140 may then determine the expected use levelsof the vehicle 108 in the relevant conditions, autonomous operationfeature settings, and/or autonomous communication levels to facilitatedetermining a total risk for the vehicle 108. The server 140 maydetermine expected vehicle use based upon the use information receivedat block 1004, including expected levels of autonomous communicationunder a plurality of sets of parameters. For example, the server 140 maydetermine based upon past operating data that the 50% of the totaloperating time of the vehicle 108 is likely to occur in conditions whereapproximately a quarter of the vehicles utilize autonomous communicationfeatures, 40% of the total operating time is likely to occur inconditions where a negligible number of vehicles utilize autonomouscommunication features, and/or 10% is likely to occur in conditionswhere approximately half of vehicles utilize autonomous communicationfeatures. Of course, each of the categories in the preceding example maybe further divided by other conditions, such as traffic levels, weather,average vehicle speed, presence of pedestrians, location, autonomousoperation feature settings, and/or other parameters. Thesedeterminations may be used to further determine the total riskassociated with autonomous vehicle operation including autonomouscommunication.

At block 1012, the server 140 may use the risk levels determined atblock 1010 to determine a total expected risk level for the vehicle 108including one or more autonomous communication features, in a similarmanner to the determination described above in block 810. The server 140may weight each of the risk levels corresponding to sets of parametersby the expected use levels corresponding to the same set of parameters.The weighted risk levels may then be aggregated using known techniquesto determine the total risk level. As above, the total risk level may beadjusted or normalized, or it may be used to determine a risk categoryor risk class in accordance with regulatory requirements.

At block 1014, the server 140 may determine one or more premiums forvehicle insurance policies covering the vehicle 108 based upon the totalrisk level determined at block 1012. As in methods 800 and/or 900,additional factors may be included in the determination of the policypremiums, and/or the premiums may be adjusted based upon additionalfactors. The server 140 may further record the premiums and/or maytransmit one or more of the policy premiums to relevant parties.

In any of the preceding embodiments, the determined risk level orpremium associated with one or more insurance policies may be presentedby the server 140 to a customer or potential customer as offers for oneor more vehicle insurance policies. The customer may view the offeredvehicle insurance policies on a display such as the display 202 of themobile device 110, select one or more options, and/or purchase one ormore of the vehicle insurance policies. The display, selection, and/orpurchase of the one or more policies may be facilitated by the server140, which may communicate via the network 130 with the mobile device110 and/or another computer device accessed by the user.

Additionally, or alternatively, any of the preceding embodiments maydetermine or adjust vehicle insurance coverage types or levels (e.g.,deductibles, coverage amounts, etc.) based upon use conditions and/orautonomous (and/or semi-autonomous) operation feature use,configuration, or settings. For example, deductibles or premiums for alevel of vehicle insurance coverage for theft of a vehicle may bereduced for policies where a fully autonomous vehicle includesautonomous operation features capable of returning the vehicle ifstolen. As another example, coverage levels of a vehicle insurancepolicy may vary based upon whether an autonomous vehicle contains anypassengers or vehicle operators. Additionally, coverage types or levelsmay be adjusted based upon use levels of the autonomous operationfeatures and/or information regarding a vehicle operator. For example,certain coverage types or levels may be unavailable to a vehicleoperator (e.g., inexperienced drivers, drivers with extensive accidenthistories, etc.), except that such coverage may be available whencertain autonomous operation features are enabled or activated. In someembodiments, vehicle operators who may be otherwise unable or legallyprevented from operating a vehicle (e.g., due to blindness, physicaldisabilities, revocation of an operating license, etc.) may be insuredfor operation of an autonomous vehicle with certain autonomous operationfeatures enabled.

Exemplary Methods of Providing Insurance Coverage

In one aspect, a computer-implemented method of adjusting or creating aninsurance policy may be provided. The method may include: (1) capturingor gathering data, via a processor, to determine an autonomous orsemi-autonomous technology or functionality associated with a specificvehicle; (2) comparing the received data, via the processor, to a storedbaseline of vehicle data created from (a) actual accident data involvingautomobiles equipped with the autonomous or semi-autonomous technologyor functionality, and/or (b) autonomous or semi-autonomous vehicletesting; (3) identifying (or assessing) accident or collision risk, viathe processor, based upon an ability of the autonomous orsemi-autonomous technology or functionality associated with the specificvehicle to make driving decisions and/or avoid or mitigate crashes; (4)adjusting or creating an insurance policy, via the processor, based uponthe accident or collision risk identified that is based upon the abilityof the autonomous or semi-autonomous technology or functionalityassociated with the specific vehicle; and/or (5) presenting on a displayscreen, or otherwise providing or communicating, all or a portion of(such as a monthly premium or discount) the insurance policy adjusted orcreated to a potential or existing customer, or an owner or operator ofthe specific vehicle equipped with the autonomous or semi-autonomoustechnology or functionality, for review, acceptance, and/or approval.The method may include additional, fewer, or alternative steps oractions, including those discussed elsewhere herein.

For instance, the method may include evaluating, via the processor, aneffectiveness of the autonomous or semi-autonomous technology orfunctionality, and/or an associated artificial intelligence, in a testenvironment, and/or using real driving experience or information.

The identification (or assessment) of accident or collision riskperformed by the processor may be dependent upon the extent of controland/or decision making that is assumed by the specific vehicle equippedwith the autonomous or semi-autonomous technology or functionality,rather than the human driver. Additionally or alternatively, theidentification (or assessment) of accident or collision risk may bedependent upon (a) the ability of the specific vehicle to use externalinformation (such as vehicle-to-vehicle, vehicle-to-infrastructure,and/or infrastructure-to-vehicle wireless communication) to make drivingdecisions, and/or (b) the availability of such external information,such as may be determined by a geographical region (urban or rural)associated with the specific vehicle or vehicle owner.

Information regarding the autonomous or semi-autonomous technology orfunctionality associated with the specific vehicle, includingfactory-installed hardware and/or versions of computer instructions, maybe wirelessly transmitted to a remote server associated with aninsurance provider and/or other third party for analysis. The method mayinclude remotely monitoring an amount or percentage of usage of theautonomous or semi-autonomous technology or functionality by thespecific vehicle, and based upon such amount or percentage of usage, (a)providing feedback to the driver and/or insurance provider via wirelesscommunication, and/or (b) adjusting insurance policies or premiums.

Data Acquisition

In one aspect, the present embodiments may relate to data acquisition.Data may be gathered via devices employing wireless communicationtechnology, such as Bluetooth or other IEEE communication standards. Inone embodiment, a Bluetooth enabled smartphone or mobile device, and/oran in-dash smart and/or communications device may collect data. The dataassociated with the vehicle, and/or vehicle or driver performance, thatis gathered or collected at, or on, the vehicle may be wirelesslytransmitted to a remote processor or server, such as a remote processoror server associated with an insurance provider. The mobile device 110may receive the data from the on-board computer 114 or the sensors 120,and may transmit the received data to the server 140 via the network130, and the data may be stored in the database 146. In someembodiments, the transmitted data may include real-time sensor data, asummary of the sensor data, processed sensor data, operating data,environmental data, communication data, or a log such data.

A. Vehicle Decision Making

Data may be generated by autonomous or semi-autonomous vehicles and/orvehicle mounted sensors (or smart sensors), and then collected byvehicle mounted equipment or processors, including Bluetooth devices,and/or an insurance provider remote processor or server. The datagathered may be used to analyze vehicle decision making. A processor maybe configured to generate data on what an autonomous or semi-autonomousvehicle would have done in a given situation had the driver not takenover manual control/driving of the vehicle or alternative controlactions not taken by the autonomous or semi-autonomous operationfeatures. This type of control decision data (related to vehicledecision making) may be useful with respect to analyzing hypotheticalsituations.

In one embodiment, an application, or other computer or processorinstructions, may interact with a vehicle to receive and/or retrievedata from autonomous or semi-autonomous processors and sensors. The dataretrieved may be related to radar, cameras, sensor output, computerinstructions or application output. Other data related to a smartvehicle controller, car navigation unit information (including routehistory information and typical routes taken), GPS unit information,odometer and/or speedometer information, and smart equipment data mayalso be gathered or collected. The application and/or other computerinstructions may be associated with an insurance provider remoteprocessor or server.

The control decision data may further include information regardingcontrol decisions generated by one or more autonomous operation featureswithin the vehicle. The operating data and control decision datagathered, collected, and/or acquired may facilitate remote evaluationand/or analysis of what the autonomous or semi-autonomous vehicle was“trying to do” (brake, slow, turn, accelerate, etc.) during operation,as well as what the vehicle actually did do. The data may revealdecisions, and the appropriateness thereof, made by the artificialintelligence or computer instructions associated with one or moreautonomous or semi-autonomous vehicle technologies, functionalities,systems, and/or pieces of equipment. The data may include informationrelated to what the vehicle would have done in a situation if the driverhad not taken over (beginning manual vehicle control). Such data mayinclude both the control actions taken by the vehicle and controlactions the autonomous or semi-autonomous operation features would havecaused the vehicle to take. Thus, in some embodiments, the controldecisions data may include information regarding control decisions notimplemented by the autonomous operation features to control the vehicle.This may occur when an autonomous operation feature generates a controldecision or associated control signal, but the control decision orsignal is prevented from controlling the vehicle because the autonomousfeature or function is disabled, the control decision is overridden bythe vehicle operator, the control signal would conflict with anothercontrol signal generated by another autonomous operation feature, a morepreferred control decision is generated, or an error occurs in theon-board computer 114 or the control system of the vehicle.

For example, a vehicle operator may disable or constrain the operationof some or all autonomous operation features, such as where the vehicleis operated manually or semi-autonomously. The disabled or constrainedautonomous operation features may, however, continue to receive sensordata and generate control decision data that is not implemented.Similarly, one or more autonomous operation features may generate morethan one control decision in a relevant period of time as alternativecontrol decisions. Some of these alternative control decisions may notbe selected by the autonomous operation feature or an autonomousoperation control system to control the vehicle. For example, suchalternative control decisions may be generated based on different setsof sensor or communication data from different sensors 120 or include orexcluding autonomous communication data. As another example, thealternative control decisions may be generated faster than they can beimplemented by the control system of the vehicle, thus preventing allcontrol decisions from being implemented.

In addition to control decision data, other information regarding thevehicle, the vehicle environment, or vehicle operation may be collected,generated, transmitted, received, requested, stored, or recorded inconnection with the control decision data. As discussed elsewhereherein, additional operating data including sensor data from the sensors120, autonomous communication data from the communication component 122or the communication module 220, location data, environmental data, timedata, settings data, configuration data, and/or other relevant data maybe associated with the control decision data. In some embodiments, adatabase or log may store the control decision data and associatedinformation. In further embodiments, the entries in such log or databasemay include a timestamp indicating the date, time, location, vehicleenvironment, vehicle condition, autonomous operation feature settings,and/or autonomous operation feature configuration information associatedwith each entry. Such data may facilitate evaluating the autonomous orsemi-autonomous technology, functionality, system, and/or equipment inhypothetical situations and/or may be used to calculate risk, and inturn adjust insurance policies, premiums, discounts, etc.

B. Evaluating Risk

The data gathered may be used to evaluate risk associated with theautonomous or semi-autonomous operation feature or technology at issue.As discussed elsewhere herein, information regarding the operation ofthe vehicle may be monitored or associated with test data or actual lossdata regarding losses associated with insurance policies for othervehicles having the autonomous technology or feature to determine risklevels and/or risk profiles. Specifically, the control decision data,sensor data, and other operating data discussed above may be used todetermine risk levels, loss models, and/or risk profiles associated withone or more autonomous or semi-autonomous operation features. Externaldata may further be used to determine risk, as discussed below. Suchdetermined risk levels may further be used to determine insurance rates,premiums, discounts, or costs as discussed in greater detail below.

In one embodiment, the data gathered may be used to determine an averagedistance to another vehicle ahead of, and/or behind, the vehicle duringnormal use of the autonomous or semi-autonomous vehicle technology,functionality, system, and/or equipment. A safe driving distance toother vehicles on the road may lower the risk of accident.

The data gathered may also relate to how quickly the technology,functionality, system, and/or equipment may properly stop or slow avehicle in response to a light changing from green to yellow, and/orfrom yellow to red. Timely stopping at traffic lights may alsopositively impact risk of collision.

The data gathered may indicate issues not entirely related to theautonomous or semi-autonomous technology, functionality, system, and/orequipment. For instance, tires spinning and low vehicle speed may bemonitored and identified to determine that vehicle movement was beingaffected by the weather (as compared to the technology, functionality,system, and/or equipment during normal operation). Vehicle tires mayspin with little or no vehicle movement in snow, rain, mud, ice, etc.

The data gathered may indicate a current version of artificialintelligence or computer instructions that the autonomous orsemi-autonomous system or equipment is utilizing. A collision riskfactor may be assigned to each version of computer instructions. Theinsurance provider may then adjust or update insurance policies,premiums, rates, discounts, and/or other insurance-related items basedupon the collision risk factor and/or the artificial intelligence orcomputer instruction versions presently employed by the vehicle (and/orupgrades there to).

C. Outside Data

The decision and operating data gathered may be merged with outsidedata, such as information related to weather, traffic, construction,and/or other factors, and/or collected from sources besides the vehicle.In some embodiments, such data from outside the vehicle may be combinedwith the control decision data and other operating data discussed aboveto determine risks associated with the operation of one or moreautonomous or semi-autonomous operation features. External dataregarding the vehicle environment may be requested or received via thenetwork 130 and associated with the entries in the log or database basedon the timestamp. For example, the location, date, and time of atimestamp may be used to determine weather and traffic conditions inwhich vehicle operation occurred. Additional external data may includeroad conditions, weather conditions, nearby traffic conditions, type ofroad, construction conditions, presence of pedestrians, presence ofother obstacles, and/or availability of autonomous communications fromexternal sources. For instance, weather may impact certain autonomous orsemi-autonomous technology, functionality, system, and/or equipmentperformance, such as fog, visibility, wind, rain, snow, and/or ice.Certain autonomous or semi-autonomous functionality may have degradedperformance: (1) on ice covered roads; (2) during snow or rain, and/oron snow or rain covered roads; (3) during poor visibility conditions,such as foggy weather; (4) in “stop and go” traffic, such as during rushhour traffic, or slow moving traffic through high construction areas ordowntown areas; and/or (5) caused by other factors.

The system and method may consider the geographical area associated withthe user, or the owner or operator of a vehicle. For instance, rainmitigation functionality or technology for vehicles may be pertinent toreducing the amount of accidents and/or the severity of such accidentsin areas of high rain fall, such as the Pacific Northwest or Florida. Onthe other hand, such functionality may have less of a beneficial impacton accidents or potential accidents in desert locations, such as Nevadaor New Mexico.

Construction-related data may also be collected and analyzed.Construction-related accident avoidance and/or mitigation technology,functionality, systems, or associated equipment may be more pertinent inlarge urban areas involving significant and lengthy construction or roadconnector projects that may include frequently changing travel patternswith little notice to drivers.

D. Autonomous Vehicle Telematics

The data gathered may relate to autonomous vehicle telematics variables.From which, usage-based insurance policies, premiums, rates, discounts,rewards, and/or other insurance-related items may be estimated, asdiscussed elsewhere herein.

For instance, if sensor data indicates that automatic braking is onlyused by the driver 50% of the time, an updated or adjusted insurancepolicy, premium, rate, and/or discount may be estimated for the driver,such as by a remote processor or server associated with the insuranceprovider. A message may be wirelessly communicated to the vehicle ormobile device associated with the driver that indicates that they maysave a given amount of money on their auto insurance if they increaseusage of the automatic braking technology or functionality to a certainpercentage of time, such as up to 90% of vehicle driving time forexample. Usage of other technologies and functionalities (including thetechnologies and functionalities discussed elsewhere herein) may bemonitored, and recommended usages thereof (and associated insurancesavings) may be provided to the insured or driver for their reviewand/or approval.

Other manners of saving money on existing auto insurance coverage may beprovided to the driver via wireless communication. For instance, apercentage of time that the vehicle is in a (1) “manual” mode oroperation; (2) semi-automated, semi-automatic, or “semi-autonomous” modeor operation; and/or (3) fully automated, fully automatic, or fully“autonomous” mode or operation may be determined from vehicle sensordata that is remotely collected, such as at or by an insurance providerremote processor or server.

The insurance provider remote processor or server may determine autoinsurance discounts increases or premium reductions based upon proposedchanges to the time that the vehicle is operated in each mode, i.e.,manual, semi-autonomous, or fully autonomous. For instance, driving in asemi-autonomous, or even autonomous mode, of operation may be the safestfor a given technology or functionality and/or under certain drivingconditions (e.g., freeway driving in clear weather and moderatetraffic). The driver may be offered a reduced insurance premium or rateto increase usage of the semi-autonomous, or even autonomous, technologyor functionality, and/or to increase usage of the semi-autonomous, oreven autonomous, technology or functionality in certain drivingconditions.

Additionally or alternatively, the insurance provider may offer aplurality of separate tiers of auto insurance policies, premiums, rates,discounts, etc. For example, the insurance provider may offer threeseparate tiers. The three separate insurance tiers of premiums, rates,discounts, etc. may be based upon (a) a manual insurance rate; (b) asemi-autonomous insurance rate; and/or (c) a fully autonomous insurancerate. The manual insurance rate may be associated with manual operationof the vehicle; the semi-autonomous insurance rate may be associatedwith semi-autonomous operation of the vehicle; and/or the fullyautonomous insurance rate may be associated with autonomous operation ofthe vehicle.

Also, the data gathered may be used to provide feedback to the customeror insured. For instance, if the vehicle is presently traveling on thehighway, a recommendation or offer may be presented to the driver, suchas via wireless communication with the vehicle that indicates that ifthe driver places the vehicle into autonomous or semi-autonomous drivingmode, the risk of collision may be reduced and/or the driver may bereceive a discount, and/or lower premium on his or her auto insurance.

Other manners of potential risk reductions may also be communicated tothe driver or owner of the vehicle. For instance, recommendations and/oradjustments to insurance policies, premiums, rates, discounts, rewards,and/or other insurance-related items may be based upon drivercharacteristics or age, such as beginning or teenage drivers.

As an example, auto insurance policies, premiums, rates, discounts,rewards, and/or other insurance-related items may be adjusted, updated,or generated based upon (1) the autonomous or semi-autonomous technologyand/or functionality; (2) an amount or percentage of driver usage ofthat technology and/or functionality; and/or (3) driver characteristics.The driver characteristics that may be taken into consideration includedriver age, driver health, and/or past driving or accident history.

E. Smart Equipment

The data gathered may originate from various smart parts and/or piecesof smart equipment mounted on a vehicle, including parts configured forwired or wireless communication. For instance, a vehicle may be equippedwith smart brakes; smart tail, head, or turn lights; smart tires; etc.Each piece of smart equipment may have a wired or wireless transmitter.Each piece of smart equipment may be configured to monitor itsoperation, and/or indicate or communicate a warning to the driver whenit is not operating properly.

As an example, when a rear brake light is out, such as from faultyrepair or from normal burn out, that fact may be detected by smartvehicle functionality and the driver may be promptly notified. As aresult, the driver may be able to repair the faulty brake light beforean accident caused by the faulty brake light occurs. In anotherembodiment, the data gathered may also indicate window wipers are notoperating properly, and need to be replaced. The insurance provider mayadjust or update insurance policies, premiums, rates, discounts, and/orother insurance-related items based upon the smart equipment warningfunctionality that may alert drivers of vehicle equipment or vehiclesafety equipment (lights, brakes, etc.) that need to be replaced orrepaired, and thus may reduce collision risk.

In addition to addressing liability for collision risk, the technologymay also reduce risk of theft. For instance, stolen vehicles may betracked via on-board GPS units and wireless transmitters. Also, thebreaking and entering, and/or hot wiring, of vehicles may be moredifficult through the use of anti-hacking measures for smart vehicles orvehicles with electrical or electronic control systems. The insuranceprovider may adjust insurance premiums, rates, and/or otherinsurance-related items based upon the reduced risk of theft.

Assignment of Fault

The present embodiments may relate to the assignment of fault. Theassignment of fault may be based upon sensor data and/or other datagathered or collected from, or by, the vehicle. The assignment of faultmay impact the future rating for one or more drivers, and/or one or morevehicles equipped with one or more autonomous or semi-autonomoustechnologies, functionalities, systems, and/or pieces of equipment.

The assignment of fault determination from sensor and/or vehicle datamay relate to, and/or involve, determining who was in control of, ordriving, the vehicle at the time of the accident (such as either thehuman driver or the vehicle itself), and/or determining who was at faultor liable for the collision or accident—the human driver or the vehicle.For instance, did the vehicle give the driver enough time (e.g., half asecond) to take manual control of the vehicle before the time of impact,or was the driver not attentive enough before an accident.

The assignment of fault may include a determination of who pays theclaim associated with a vehicle accident and/or determine future ratingsfor certain types of technology or functionality, and/or certaininsurance policy holders. Fault for a vehicle collision or accident maybe partially or fully assigned to one or more drivers, and/or one ormore vehicles equipped with one or more autonomous or semi-autonomoustechnologies, functionalities, systems, and/or pieces of equipment.

In one embodiment, insurance coverage may provide for immediatecoverage/payment to an insured in the case of an accident. After which,based upon data collected from the smart vehicle or sensors, blame orfault may be assigned for the accident, such as to either the driver orthe autonomous or semi-autonomous technology or functionality, and/or toanother driver, or autonomous or semi-autonomous vehicle involved in theaccident.

Alluded to above, the data gathered may help determine who was incontrol of the vehicle before, during, and/or after a vehicle collisionor accident. For instance, a human driver, or an autonomous orsemi-autonomous vehicle (and/or associated technology, functionality,system, and/or equipment) may have been in control of the vehicle at thetime of accident. The data may be used to identify whether there wasenough time for a driver to takeover manually. For instance, once ahazardous condition is identified (e.g., vehicles slowing down abruptlyor heavy congestion ahead, or vehicle accident ahead), did asemi-autonomous technology function correctly, and/or did the humandriver have the time to take manual control of the vehicle and avoid acollision or accident.

For a fully autonomous vehicle, technology, or functionality whether ornot the collision or accident could have been avoided may be determined.For example, a performance of the artificial intelligence or computerinstructions associated with the autonomous vehicle, technology, orfunctionality may be evaluated. In accidents or collisions involvingone, two, or more autonomous vehicles, evaluating the performance of theautonomous technology or functionality may determine fault—such as whichautonomous vehicle was at fault (for an accident involving twoautonomous vehicles) or whether an autonomous vehicle or a human driverwas at fault (for an accident involving two vehicles, one driven by ahuman driver and one driven by autonomous vehicle technology,functionality, or systems that may include associated artificialintelligence and/or processors).

Insurance Adjustment Recommendations

Autonomous or semi-autonomous technology, functionality, and/or systemusage data may be used to identify and present a driver one or morepotential premium or rate reductions with increased usage of thetechnology, functionality, and/or system. A number of “what if”insurance-related scenarios may be calculated and then presented to adriver and/or insured for their review, approval, and/or modification.The different scenarios may be presented to a driver on their mobiledevice or a smart vehicle display screen, or other in dash display.

Autonomous or semi-autonomous vehicle technology or functionality mayrelate to vehicle parking. The technology or functionality may determinean available parking spot in an urban area or large city. The smartvehicle may make recommendations to the driver regarding the bestavailable parking spot remotely identified. For instance, the bestavailable parking spot may be determined based upon the cost of theparking; safety of the parking spot, lot, or garage; the risk of theftor other liability associated with the parking spot, lot, or garage;and/or other factors.

The recommendation may be generated by an insurance provider remoteprocessor or server. The recommendation with respect to best availableparking spot may include information regarding an adjustment to thedriver's present insurance policy, premium, rate, and/or discount basedupon the driver accepting the recommendation and parking the vehicle inthe best available parking spot. A discount and/or lower premium may beoffered to the driver to encourage safer parking habits that may reducethe risk of vehicle damage or theft.

Alternatively, based upon an actual parking spot, additional insurancecoverage may be offered to the driver. For instance, if the vehicle isgoing to be parked on a busy street and overnight, it may have a higherrisk of damage or theft. A remote processor or server associated withthe insurance provider may estimate and/or offer an appropriate increasein auto insurance coverage to the insured or driver, such as viawireless communication with a smart vehicle controller or a mobiledevice of the insured or driver.

Exemplary Feedback Method

Beyond determining risk categories or premiums for vehicle insurancepolicies covering autonomous (and/or semi-autonomous) vehicles, in someembodiments the system 100 may operate to monitor use of autonomous(and/or semi-autonomous) operation features and present feedback tovehicle operators. This may occur in real time as operating conditionschange or may occur on a periodic basis in response to vehicle use andenvironmental conditions. The use of autonomous operation features maybe assessed to determine whether changes to the number, type,configuration, or settings of the autonomous operation features used mayreduce the risk associated with vehicle operation under variousconditions. Presenting or otherwise providing the information to thevehicle operator may improve the effective use of the autonomousoperation features and/or reduce the risks associated with vehicleoperation. Upon receiving a suggestion regarding autonomous operationfeature use, the vehicle operator may be able to maximize theeffectiveness of the autonomous operation feature, maximize vehicleinsurance coverage, and/or minimize vehicle insurance expense.

FIG. 11 illustrates an exemplary autonomous (and/or semi-autonomous)operation feature monitoring and feedback method 1100. The method 1100may be performed by the controller 204 or the server 140 at any timewhile the vehicle 108 is in operation. In some embodiments, the method1100 may be implemented only when the vehicle 108 is stationary, whenthe autonomous (and/or semi-autonomous) operation features arecontrolling the vehicle 108, when the controller 204 or server 140determines that the conditions meet certain criteria (e.g., when thevehicle is more than a predetermined distance from environmentalobstacles on a restricted access highway, etc.), and/or when the vehicle108 is first started, such that the method 1100 does not distract thevehicle operator. During implementation of the method 1100, thecontroller 204 may determine actual use levels of the autonomousoperation features at block 1102. This may include current use of thefeatures and/or past use of the features, either generally or undersimilar conditions. In some embodiments, the determination of use levelsmay include a determination of the use of versions, configurations, orsettings related to the autonomous operation features.

At block 1104, the controller 204 may receive sensor data from thesensors 120, as discussed above. The received sensor data may includeinformation regarding the vehicle 108, the vehicle's environment (e.g.,traffic conditions, weather conditions, etc.), and/or the vehicleoperator. The sensor data may include information regarding the physicalor mental state of the vehicle operator using sensors 120 disposedwithin the vehicle 108 or communicatively connected thereto (e.g.,disposed within or communicatively connected to a mobile device 110,such as a smart phone, and/or a wearable computing device, such as asmart watch or smart glasses). This sensor data may include data frominterior cameras, microphones, accelerometers, and/or physiologicalsensors (e.g., thermometer, microphone, thermal image capture device,electroencephalograph, galvanic skin response sensor, heart ratesensors, respiratory rate sensor, other biometric sensors, etc.). Insome embodiments, the received sensor data may exclude sensor dataregarding the vehicle operator or the physical or mental state of thevehicle operator.

At block 1106, the controller 204 or the server 140 may receivecommunication data from external sources. The communication data mayinclude direct communication data from other autonomous vehicles,communicating infrastructure, and/or other smart devices (e.g., mobiledevices carried or worn by pedestrians, passengers in other vehicles,etc.). The communication data may also include indirect communicationdata received by the controller 204 or the server 140 via the network130 (e.g., information regarding traffic, construction, accidents,weather, local time, local events, local traffic patterns, localaccident statistics, general accident statistics, etc.). The indirectinformation may be obtained from database 146 or from other networked orthird-party databases. For example, indirect communication data may beobtained regarding the risk level of autonomous operation featuresrelative to manual vehicle operation on major highways during typicalcommuting times in urban areas in light rain, which may be combined withinformation from a weather service indicating light rain and informationfrom a map service indicating the vehicle 108 is on a major highway(using GPS data from the sensors 120). As a further example, trafficdatabases could be accessed to receive information regarding accidentsand/or construction further ahead along a route.

At block 1108, the server 140 or the controller 204 may determine anoptimal use level for the autonomous operation features available withinthe vehicle 108 and/or a suggestion regarding the optimal autonomousoperation feature use level under the conditions. The optimal use levelor suggestion may include the types and versions of autonomous operationfeatures to use, configurations of the features, and/or settingsrelating to the features. The server 140 or the controller 204 maydetermine one or more optimal use levels for the autonomous operationfeatures based upon the sensor data and communication data received inblocks 1104 and 1106 using any known or later-developed optimizationtechniques. In some embodiments, the risk levels associated with eachcombination of use levels for autonomous operation features may bedetermined and stored in one or more databases, such that the server 140or controller 204 may access and compare the appropriate databaseentries to determine the optimal use levels. In further embodiments, oneor more optimal use levels may be determined and stored in one or moredatabases, such that the server 140 or controller 204 may determine theoptimal use level by accessing the database entry corresponding to thesensor data and communication data. Alternatively, the server 140 orcontroller 204 may determine optimal use levels by determining risklevels for a variety of combinations of configurations and settingsassociated with autonomous operation features based on the receivedsensor data and communication data. In such embodiments, the combinationor combinations determined to have the lowest risk may be determined tobe the optimal feature use levels.

The determination of optimal feature use may be based upon the receivedsensor data and/or the communication data. In some embodiments, thereceived sensor data and/or communication data may include informationregarding the physical, mental, and/or emotional state of the vehicleoperator, as noted above. In various embodiments, the determination ofthe optimal feature use level may either include or exclude informationregarding the state of the vehicle operator from the determination. Forexample, the determination may be based in part upon the previousdriving history of a vehicle operator, which may indicate that thevehicle operator has an increased risk of an accident in low lightenvironments. In the example, the determination may compare the expectedperformance of the various autonomous operation features against theexpected performance of the vehicle operator, which may cause the server140 or controller 204 to determine an optimal feature use level thatincludes more autonomous operation feature use than would otherwise bedetermined to be optimal. As a related example, the server 140 or thecontroller 204 may not determine the optimal use level based upon theprevious driving history of the vehicle operator from the previousexample, which may result in a determination of an optimal feature uselevel that includes less use of autonomous operation features than inthe preceding example.

The determined optimal use level may be used to further determine anautonomous (and/or semi-autonomous) operation feature use suggestion.The use suggestion may include one or more settings relating toautonomous operation features, enabling or disabling particularautonomous operation features, using specific versions of autonomousoperation features, resuming manual operation of the vehicle,temporarily ceasing autonomous and/or manual operation of the vehicle,and/or similar changes to the use and configuration of the autonomousoperation features in operating the vehicle 108. It should be noted thatthe determined use suggestion may include changes to the use ofautonomous operation features, use of additional autonomous operationfeatures, and/or use of fewer autonomous operation features.

At block 1110, the suggested optimal use levels of autonomous (and/orsemi-autonomous) operation features determined at block 1108 is comparedagainst the actual autonomous operation feature use levels determined atblock 1102. When the suggested and optimal feature use levels aredetermined to be different, the server 140 or the controller 204 causesa suggestion of autonomous operation feature use to be presented to thevehicle operator at block 1112. In some embodiments, the suggestion maynot be presented when the difference between the optimal use level andthe actual use level is below a predetermined threshold. For example,the server 140 may determine not to present the suggested autonomousoperation use to the vehicle operator where the difference would onlyresult in a risk reduction equivalent to a monetary value below fivecents.

The suggestion presented at block 1112 may be presented using a displaywithin the vehicle 108, a mobile device 110, or other means, includingvisual and/or audible notifications. The suggestion may include arecommendation that the vehicle operator enable or use one or moreadditional autonomous (and/or semi-autonomous) operation feature, thatthe vehicle operator change the settings or configuration for one ormore autonomous (and/or semi-autonomous) operation features, that thevehicle operator disable or discontinue use of one or more autonomous(and/or semi-autonomous) operation features, and/or related changes thatmay be made to the use of the autonomous (and/or semi-autonomous)operation features. The suggestion may further include one or morereasons for making a change to the autonomous operation feature use,such as an indication of a reduction in risk, a percentage reduction inthe probability of a collision, an increase in a probability ofcompleting the trip without incident, a reduction in a premium or otherpolicy charge, a reduction in a rate, an increase in a coverage amount,an increase in a coverage type, a reduction in a deductible, and/orrelated information to induce the vehicle operator to change theautonomous operation feature use. For example, a suggestion presented tothe vehicle operator may indicate that updating to a newer softwareversion of an autonomous operation feature would result in a decrease ofa certain amount in a vehicle insurance premium. In some embodiments,the vehicle operator may make a selection upon presentation of thesuggestion, which selection may cause the use levels of one or more ofthe autonomous operation features to be adjusted (e.g., to match the oneor more optimal use levels). In other embodiments, the vehicle operatormay otherwise adjust or control the use levels, as discussed above. Achange or adjustment to the use, configuration, or settings of theautonomous operation features may further cause a change or adjustmentto costs or coverage associated with a vehicle insurance policy, asdiscussed above.

After a suggestion has been presented at block 1112 or when thesuggested optimal feature use is determined to be insufficientlydifferent from the actual feature use at block 1110, the server 140 orthe controller 204 determine whether vehicle operation is ongoing atblock 1114. When operation is ongoing, the method 1100 may repeat thesteps of blocks 1102-1112. In some embodiments, the method 1100 mayrepeat only when a predetermine period of time (e.g., 5 minutes, 15minutes) has passed, when vehicle operating conditions have sufficientlychanged (e.g., upon exiting a highway, entering fog, sunset, etc.),and/or when a sufficient change in the recommendation has occurred(e.g., risk level, monetary incentive, feature use level recommendation,etc.). When the operation of the vehicle 108 is complete, the method1100 may terminate. In some embodiments, however, the method 1100 may beimplemented either before or after vehicle operation, in which case theactual autonomous (and/or semi-autonomous) operation feature usedetermined in block 1102 may be based upon the settings of theautonomous operation features that had been previously used, thesettings that would be applied if the vehicle were to be used at thattime, or the default settings.

Exemplary Warning Method

In addition to monitoring use of autonomous operation features topresent feedback regarding autonomous (and/or semi-autonomous) operationfeature use to vehicle operators, some embodiments may determineelevated risk levels and present warnings to the vehicle operator. Insome embodiments, this may include warnings regarding situations whereno changes to the optimal autonomous operation feature use level wouldbe suggested, but where an increased risk nonetheless exists. Forexample, communication data regarding recent snowfall may be combinedwith sensor data indicating a high frequency of slipping wheels todetermine a high risk of an accident exists at the current speed on asnow-covered road. The vehicle operator might then respond by reducingthe speed of the vehicle, resuming manual control of the vehicle, and/orselecting an alternate route using major thoroughfares that are clear ofsnow. Such responses may further cause an adjustment in a cost orcoverage level associated with a vehicle insurance policy.

FIG. 12 illustrates an exemplary autonomous (and/or semi-autonomous)operation feature monitoring and alert method 1200. The method 1200 maybe performed by the controller 204 or the server 140 at any time whilethe vehicle 108 is in operation. During implementation of the method1200, the controller 204 or server 140 may determine the use of theautonomous operation features at block 1202. This may include currentuse of versions, configurations, or settings related to the autonomousoperation features. As discussed above, the controller 204 or server 140may further receive sensor data and communication data, respectively, atblocks 1204 and 1206. The sensor data may be received from sensors 120disposed within the vehicle 108, and the communication data may includeinformation regarding the vehicle environment (including informationregarding the rate of incidents in similar conditions or locations basedon historical data). This information may be used at block 1208 todetermine the risk associated with operation of the vehicle under theconditions. As above, the sensor data, communication data, and thedetermination of risk may either include or exclude informationregarding one or more vehicle operators (e.g., the physical, mental,and/or emotional state of the vehicle operator).

At block 1208, the server 140 or the controller 204 may determine a risklevel associated with the operation of the vehicle under the currentconditions. This may include a determination of the risk associated withthe autonomous (and/or semi-autonomous) vehicle operation features thenin use, or it may include a determination of the risk associated withvarious configurations or settings of autonomous operation features asdiscussed above with respect to method 1100. In some embodiments, thedetermination may not include information regarding one or more vehicleoperators. The server 140 or controller 204 may determine one total risklevel or a plurality of risk levels associated with vehicle operation atblock 1208. For example, separate risk levels may be determined fordifferent types of potential incidents (e.g., collisions with othervehicles, loss of control or traction, collisions with pedestrians,collisions with stationary obstructions, etc.).

At block 1210, the server 140 or the controller 204 may compare thedetermined risk level against a warning threshold risk level. In someembodiments, the difference between the determined risk level and a risklevel associated with an optimal autonomous (and/or semi-autonomous)operation feature use level (as discussed above with respect to method1100) may be compared against the warning threshold, and the warningthreshold may be set at a level such that a warning is triggered onlywhen the additional risk from suboptimal autonomous (and/orsemi-autonomous) operation feature use, configuration, and/or settingsexceeds a certain level. In further embodiments, the risk level may becompared against a plurality of predetermined warning thresholds, andthe warning presented to the vehicle operator may be determined basedupon the highest warning threshold exceeded by the risk level.

When the risk level is determined to exceed the warning threshold atblock 1210, the controller 204 or server 140 may cause a warning to bepresented to the vehicle operator at block 1212. The warning presentedat block 1212 may be presented using a display within the vehicle 108, amobile device 110, or other means, including visual, audible, and/orhaptic notifications. The warning may specify one or more causes of theelevated risk (e.g., weather, speed, hardware malfunctions, etc.).Alternatively, the warning may simply alter the vehicle operator to anelevated risk level. In some embodiments, the vehicle operator may makea selection upon presentation of the alert, which selection may causethe use, configuration, or settings of one or more of the autonomous(and/or semi-autonomous) operation features to be adjusted (e.g., thevehicle operator may resume full control of operation, the vehicleoperator may cede control of operation to the autonomous (and/orsemi-autonomous) operation features, etc.). In other embodiments, thevehicle operator may otherwise adjust or control the use levels, asdiscussed above. A change or adjustment to the use, configuration, orsettings of the autonomous operation features may further cause a changeor adjustment to costs or coverage associated with a vehicle insurancepolicy, as discussed above.

After the warning has been presented at block 1212 or when the risklevel is determined to be below the risk threshold at block 1210, theserver 140 or the controller 204 determine whether vehicle operation isongoing at block 1214. When operation is ongoing, the method 1200 mayrepeat the steps of blocks 1202-1212. When the operation of the vehicle108 is complete, the method 1200 may terminate.

Exemplary Fault Determination Method

In some embodiments, the system 100 may be used to determine or allocatefault upon the occurrence of an accident or other collision involvingthe vehicle 108. Information regarding the operation of the vehicle 108may be recorded and stored during operation, which may then be used todetermine the cause of a collision or accident automatically uponreceiving an indication of the occurrence of such. Fault may beallocated to either the vehicle operator, one or more autonomousoperation features, or a third party (e.g., another motorist orautonomous vehicle). Such allocation of fault may be further used toadjust one or more of an insurance policy premium, a risk level, a ratecategory, a penalty, or a discount relating to a vehicle insurancepolicy. In some embodiments, the allocation of fault may also be used todetermine whether to cancel an insurance policy, adjust a deductible,adjust a policy limit, and/or determine a payment associated with thecollision or accident. Where an autonomous operation feature isdetermined to be wholly or partially responsible for the accident, therisk levels or risk profile associated with that autonomous operationfeature may be revised, such that the risk levels or risk profile ofother autonomous vehicles using the feature may also be adjusted.

FIG. 13 illustrates an exemplary fault determination method 1300 fordetermining fault following an accident based upon sensor data andcommunication data. Upon receiving an indication of an accident at block1302, the method 1300 may receive sensor data and communication data atblock 1304 and may further receive information regarding the operationof one or more autonomous (and/or semi-autonomous) operation features atblock 1306. In some embodiments, this information may be used to make apreliminary determination of whether a third party is at fault at block1308, in which case there may be no fault allocated to the vehicleoperator and/or autonomous (and/or semi-autonomous) operating features.If a third party was not at fault or if the vehicle 108 had the lastchance to avoid the accident, the method 1300 may then determine andallocate fault between the vehicle operator and one or more autonomous(and/or semi-autonomous) operation features in blocks 1314-1324. Thedetermination of fault may further be used to determine and/or adjust acoverage level at block 1326, such as a deductible level or a policystatus.

The determination process of method 1300 may depend upon whether thevehicle 108 is operated in a fully autonomous, partially autonomous, ormanual operation mode at the time of the accident. In some embodiments,the server 140 may determine and/or allocate fault without humaninvolvement. In other embodiments, the server 140 may present relevantinformation and/or a determination of fault to a reviewer (e.g., aclaims adjuster or other specialist) for verification or furtheranalysis. In such embodiments, the presented information may includesummaries or detailed reports of sensor data and/or communication data,including still images or video recordings from the sensors 120 withinthe vehicle 108 or other sensors at the location of the accident (e.g.,sensors disposed within other vehicles involved in or near the accidentsite, sensors disposed within infrastructure elements, etc.). The method1300 may be implemented by the mobile device 110, the on-board computer114, the server 140, and/or some combination of these components.

At block 1302, the server 140 may receive an indication of an accidentinvolving the vehicle 108. The server 140 or controller 204 may generatethis indication automatically based on sensor data, or it may beinitiated manually by a vehicle operator or another person following theaccident. However the indication is received, it may cause the method1300 to proceed to the one or more determinations of fault.

At block 1304, the server 140 may receive sensor data from the one ormore sensors 120 within the vehicle 108 and/or communication data fromthe communication component 122 and/or the communication unit 220. Inaddition, the server 140 may receive additional information fromexternal sources, including sensor data from other vehicles orinfrastructure, communication information from other vehicles orinfrastructure, and/or communication information from third-partysources. For example, additional information may be obtained from otherautonomous vehicles involved in the accident or near the accident. Asdiscussed above, the server 140 may additionally receive controldecision data regarding the control decisions generated by one or moreof the autonomous operation features of the vehicle 108. In someembodiments, the sensor and/or communication data may be stored in thedatabase 146 or in the program memory 160 or 208, and/or in the RAM 164or 212 during ordinary operation of the vehicle 108, from which the datamay be retrieved or accessed by the server 140. Additionally, oralternatively, the sensor and/or communication data may be stored inanother memory or database communicatively connected to the network 130.In some embodiments, a back-up of the sensor and/or communication datamay be stored within a memory (not shown) that may be designed towithstand the forces and temperatures frequently associated with avehicular collision.

At block 1306, the server 140 may further receive information regardingthe operation of the autonomous (and/or semi-autonomous) operationfeatures in the vehicle 108. This information may include informationregarding use, configuration, and settings of the features concurrentwith the accident. In some embodiments, the information may furtherinclude information regarding control signals or outputs from theautonomous operation features to control the vehicle 108. This may beuseful, for example, in determining whether the autonomous operationfeature failed to take appropriate control actions or whether thecontrol signals were not implemented or were ineffective in controllingthe vehicle 108 (e.g., such as may occur when on ice or when a defectprevents an electromechanical control from properly functioning). Insome embodiments, autonomous operation feature data may be available foradditional vehicles involved in the accident, which may be accessed orobtained by the server 140. As above, the autonomous operation featuredata may be recorded during ordinary operation of the vehicle 108 andaccessed or obtained by the server 140 upon receipt of the indication ofthe accident.

At block 1308, the server 140 may determine whether a third party is atfault for the accident based upon the sensor data, communication data,and/or autonomous (and/or semi-autonomous) operation feature datareceived in blocks 1304 and 1306. Determining fault may generallyinclude determining one or more of the following: a point of impact onthe vehicle 108, a point of impact on one or more additional vehicles, avelocity of the vehicle 108, a velocity of one or more additionalvehicles, a movement of the vehicle 108, a movement of one or moreadditional vehicles, a location of one or more obstructions, a movementof one or more obstructions, a location of one or more pedestrians, amovement of one or more pedestrians, a measure of road surfaceintegrity, a measure of road surface friction, a location of one or moretraffic signs or signals (e.g., yield signs, stop signs, traffic lights,etc.), an indication of the state of one or more traffic signs orsignals, a control signal generated by one or more autonomous operationfeatures of the vehicle 108, and/or a control signal generated by one ormore autonomous operation features of one or more additional vehicles.Based upon the above-mentioned factors, the server 140 may determinewhether the vehicle 108 (including the vehicle operator and/or theautonomous operation features) caused the accident or whether a thirdparty (including other autonomous vehicles, other vehicle operators,pedestrians, etc.) caused the accident.

For purposes of determining fault at block 1310, in some embodiments theserver 140 may include unavoidable accidents as being the fault of athird party (e.g., a bridge collapse, an animal suddenly darting intothe path of a vehicle, etc.). Additionally, or alternatively, physicaldefects in the autonomous vehicle 108 or the physical components of theautonomous operation features (e.g., the sensors 120, the on-boardcomputer 114, or connections within the vehicle 108) may be determinedby the server 140 as being the fault of a third party (e.g., the vehiclemaker, the original equipment manufacturer, or the installer).

When the accident is determined at block 1310 to have been caused by athird party, the server 140 may then determine whether the vehicle 108or the vehicle operator had a chance to avoid the accident that was nottaken at block 1312. For example, the vehicle operator may have beenable to avoid a collision by braking or swerving but for inattentivenessat the time of the accident. Where no such chance for the vehicleoperator or the autonomous operation features to avoid the accident isdetermined to have existed at block 1312, the fault determination method1300 may terminate. Where such a chance to avoid the accident isdetermined to have existed at block 1312, the method 1300 may continueto allocate a portion of the fault between the vehicle operator and theautonomous operation features.

At block 1314, the server 140 may determine the operating control statusof the vehicle 108 at the time of the accident based upon the receivedautonomous (and/or semi-autonomous) operation feature data regarding theuse, configuration, and settings of the features. The vehicle 108 may bedetermined to have been either manually, fully autonomously, orpartially autonomously operated at the time of the accident. Based uponthe determination, the allocation of fault will be determineddifferently. Of course, any allocation of fault to a third party aboveat block 1310 may decrease the total fault to be allocated between thevehicle operator and the one or more autonomous operation features.

Where it is determined at block 1314 that the vehicle 108 was operatingentirely manually without any autonomous operation features at the timeof the accident, the fault may be allocated entirely to the vehicleoperator. In such case, the server 140 may adjust (or cause to beadjusted) the risk or rate profile associated with the vehicle operatorat block 1322 in a manner similar to the adjustment that is typicallymade when a vehicle operator of a non-autonomous vehicle is determinedto be at fault for an accident.

Where it is determined at block 1314 that the vehicle 108 was operatingin a fully autonomous mode at the time of the accident, the fault willusually be assigned entirely to one or more autonomous operationfeatures. There are some situations, however, where the autonomousoperation feature may recognize a situation where autonomous operationis no longer feasible due to conditions in the vehicle's environment(e.g., fog, manual traffic direction, etc.). When it is determined thatthe vehicle 108 was operating as a fully autonomous vehicle at block1314, therefore, the server 140 may determine whether the one or moreautonomous operation features attempted to return control of the vehicleto the vehicle operator prior to the accident at block 1318. Becausesuch attempts may require the vehicle operator to be alert and capableof receiving control from the autonomous operation features, an adequateperiod of time for transition may be required. Thus, when it isdetermined at block 1320 that the autonomous operation features did notattempt to return control of the vehicle 108 to the vehicle operator orfailed to provide sufficient time to transfer control, the server 140may allocate fault for the accident to the one or more autonomousoperation features and adjust the risk levels and/or risk profilesassociated with the one or more autonomous operation features at block1324. When it is instead determined that the autonomous operationfeatures attempted to return control of the vehicle 108 to the vehicleoperation with adequate time for transferring control at block 1320, theserver 140 may allocate fault to the vehicle operator, and the vehicleoperator's risk or rate profile may be adjusted at block 1322. In someembodiments, the server 140 may allocate some portion of the fault toeach of the vehicle operator and the autonomous operation features wherean attempt to return control of the vehicle 108 to the vehicle operatorwas made, notwithstanding vehicle operator inattention.

Where it is determined at block 1314 that the vehicle 108 was operatingin a partially autonomous mode at the time of the accident, the server140 determines an allocation of fault between the vehicle operator andone or more autonomous operation features at block 1316. Thisdetermination may include determining which autonomous operationfeatures were in use at the time of the accident, the settings of thoseautonomous operation features, and whether the vehicle operator overrodethe operation of the autonomous operation features. For example, theserver 140 may determine that an autonomous operation feature such asadaptive cruise control without lane centering to be fully or primarilyresponsible for an accident caused by the vehicle 108 striking anothervehicle directly ahead in the same lane. In contrast, the server 140 maydetermine the vehicle operator to be fully or primarily at fault whenthe same adaptive cruise control without lane centering was engaged whenthe vehicle 108 struck another vehicle in an adjacent lane. Upondetermining the allocation of fault at block 1316, the server 140 mayadjust the vehicle operator and/or autonomous operation feature risklevels accordingly in blocks 1322 and/or 1324, respectively. In someembodiments, the use of autonomous operation features may be consideredin reducing the adjustment to the vehicle operator risk or rate profile,thereby mediating the impact of the accident on the rates or premiumsassociated with vehicle insurance for the vehicle operator.

At block 1326 the method 1300 may further utilize the faultdeterminations for underwriting and/or claim administration, in additionto or as an alternative to adjusting one or more risk levels associatedwith the vehicle operator or the autonomous operation features. Theserver 140 may further determine and/or adjust one or more coveragelevels associated with an insurance policy covering the vehicle 108based upon the allocation of fault between the vehicle operator and theautonomous operation features. For example, coverage levels may increaseor decrease as the portion of the fault allocated to the autonomousoperation feature increases or decreases. In some embodiments, thecoverage level may be associated with the accident, such as adeductible, an estimate of a cost to repair or replace the vehicle, anestimate of a cost to repair or replace other property, and/or otherpayments or adjustments to payments associated with damage or injuriesarising from the specific accident. In further embodiments, the coveragelevel may be associated with a general aspect of the insurance policy,such as a type of coverage, a maximum payment with respect to one ormore types of coverage, a maximum payment per person, and/or a maximumtotal payment per accident. For example, where an autonomous operationfeature is determined to be at fault at blocks 1314-1324, a deductibleassociated with the insurance policy may be reduced or eliminated withrespect to the accident. Where both the vehicle operator and anautonomous operation feature are determined to be partially at fault,the coverage level may be determined or adjusted based upon the portionof the fault allocated to each, either directly proportionally orotherwise. For example, the deductible for an accident caused in equalpart by the vehicle operator and one or more autonomous operationfeatures may be reduced by some amount (e.g., 25%, 50%, 75%). As afurther example, some or all of the maximum coverage limits associatedwith the insurance policy may be increased (e.g., 25% increase inliability coverage, 50% increase in collision coverage, etc.).

In some embodiments, the determination and/or adjustment of the coveragelevels may include a determination of ongoing coverage levels orcoverage status. The server 140 may determine whether to cancel ordecline to renew a policy based upon the allocation of fault and/orreceived information regarding the accident. For example, when thevehicle operator is determined to be wholly at fault or to be above afault proportion threshold (e.g., the vehicle operator is responsiblefor 80% of the fault allocated), the server 140 may determine to cancelthe policy in accordance with its terms. Alternatively, the server 140may determine to require or exclude certain types or levels of coverage,either generally or in a manner dependent upon the settings orconfiguration of one or more of the autonomous operation features. Forexample, when the vehicle operator is determined to be above a faultproportion threshold, the server 140 may limit renewal options toexclude collision coverage or limit the maximum policy limit ofcollision coverage. In some embodiments, coverage may further be limitedor cancelled based upon the use of specific autonomous operationfeatures, such as where an autonomous operation feature is determined tobe wholly or partially at fault. Determinations regarding coveragelevels or coverage status may be further based upon informationregarding the vehicle and/or the vehicle operator, such as informationregarding previous accidents and/or the vehicle operator's history ofuse of autonomous operation features (including settings used). Furtherembodiments may require or exclude coverage (or levels of coverage)based upon future use of autonomous operation features. For example,following an accident caused by the vehicle operator driving at night,the server 140 may determine to cancel coverage for future claimsarising from manual operation of the vehicle by the vehicle operator atnight. Thus, the coverage may only extend to operation of the vehicleduring daylight hours or autonomous operation of the vehicle at night.

In some embodiments, the server 140 may cause one or more of theadjustments to the insurance policy to be presented to the vehicleoperator or other customer. The customer may be presented with one ormore options relating to adjustments to the policy (e.g., options toeither agree to use autonomous operation features with certain settingsor to forego a type of coverage). Where the vehicle operator declinesall presented options, the server 140 may cancel or decline to renew thepolicy. As with determination of fault, in some embodiments,determinations of adjustments or cancellation may be presented to areviewer (e.g., a claims adjuster or other specialist) for verificationor further analysis prior to becoming effective.

Once the server 140 has assigned fault, adjusted the vehicle operator'srisk or rate profile and/or one or more of the autonomous (and/orsemi-autonomous) operation feature risk levels or profiles, anddetermined or adjusted one or more coverage levels associated with thevehicle insurance policy, the fault determination method 1300 mayterminate. The adjusted risk levels or profiles may be used to adjust apremium, surcharge, penalty, rate, or other cost associated with avehicle insurance policy for the vehicle 108 and/or the vehicleoperator.

In some embodiments, the fault determination method 1300 may beimplemented after payment has been made on claims relating to theaccident. Because the sensor, communication, and autonomous operationfeature data may be stored for later use, as discussed above, paymentmay be made shortly after occurrence of the accident. Determination offault may then be made or verified at a later date. For example,operating data concerning an accident may be stored for later usefollowing the accident, but payment of claims based upon a vehicleinsurance policy covering the vehicle may be made before a determinationof fault. Alternatively, or additionally, the fault determination method1300 may be used to preliminarily determine fault immediately or shortlyafter the occurrence of an accident, and payment of claims may be madebased upon such preliminary determination. Review and assessment of thepreliminary determination may be completed at a later time, therebyallowing faster processing of claims.

Autonomous Vehicle Insurance Policies

The disclosure herein relates to insurance policies for vehicles withautonomous operation features. Accordingly, as used herein, the term“vehicle” may refer to any of a number of motorized transportationdevices. A vehicle may be a car, truck, bus, train, boat, plane,motorcycle, snowmobile, other personal transport devices, etc. Also asused herein, an “autonomous operation feature” of a vehicle means ahardware or software component or system operating within the vehicle tocontrol an aspect of vehicle operation without direct input from avehicle operator once the autonomous operation feature is enabled orengaged. Autonomous operation features may include semi-autonomousoperation features configured to control a part of the operation of thevehicle while the vehicle operator control other aspects of theoperation of the vehicle. The term “autonomous vehicle” means a vehicleincluding at least one autonomous operation feature, includingsemi-autonomous vehicles. A “fully autonomous vehicle” means a vehiclewith one or more autonomous operation features capable of operating thevehicle in the absence of or without operating input from a vehicleoperator. Operating input from a vehicle operator excludes selection ofa destination or selection of settings relating to the one or moreautonomous operation features.

Additionally, the term “insurance policy” or “vehicle insurance policy,”as used herein, generally refers to a contract between an insurer and aninsured. In exchange for payments from the insured, the insurer pays fordamages to the insured which are caused by covered perils, acts, orevents as specified by the language of the insurance policy. Thepayments from the insured are generally referred to as “premiums,” andtypically are paid by or on behalf of the insured upon purchase of theinsurance policy or over time at periodic intervals. Although insurancepolicy premiums are typically associated with an insurance policycovering a specified period of time, they may likewise be associatedwith other measures of a duration of an insurance policy, such as aspecified distance traveled or a specified number of trips. The amountof the damages payment is generally referred to as a “coverage amount”or a “face amount” of the insurance policy. An insurance policy mayremain (or have a status or state of) “in-force” while premium paymentsare made during the term or length of coverage of the policy asindicated in the policy. An insurance policy may “lapse” (or have astatus or state of “lapsed”), for example, when the parameters of theinsurance policy have expired, when premium payments are not being paid,when a cash value of a policy falls below an amount specified in thepolicy, or if the insured or the insurer cancels the policy.

The terms “insurer,” “insuring party,” and “insurance provider” are usedinterchangeably herein to generally refer to a party or entity (e.g., abusiness or other organizational entity) that provides insuranceproducts, e.g., by offering and issuing insurance policies. Typically,but not necessarily, an insurance provider may be an insurance company.The terms “insured,” “insured party,” “policyholder,” and “customer” areused interchangeably herein to refer to a person, party, or entity(e.g., a business or other organizational entity) that is covered by theinsurance policy, e.g., whose insured article or entity is covered bythe policy. Typically, a person or customer (or an agent of the personor customer) of an insurance provider fills out an application for aninsurance policy. In some cases, the data for an application may beautomatically determined or already associated with a potentialcustomer. The application may undergo underwriting to assess theeligibility of the party and/or desired insured article or entity to becovered by the insurance policy, and, in some cases, to determine anyspecific terms or conditions that are to be associated with theinsurance policy, e.g., amount of the premium, riders or exclusions,waivers, and the like. Upon approval by underwriting, acceptance of theapplicant to the terms or conditions, and payment of the initialpremium, the insurance policy may be in-force, (i.e., the policyholderis enrolled).

Although the exemplary embodiments discussed herein relate to automobileinsurance policies, it should be appreciated that an insurance providermay offer or provide one or more different types of insurance policies.Other types of insurance policies may include, for example, commercialautomobile insurance, inland marine and mobile property insurance, oceanmarine insurance, boat insurance, motorcycle insurance, farm vehicleinsurance, aircraft or aviation insurance, and other types of insuranceproducts.

Analyzing Effectiveness of Technology & Functionality

In one aspect, the present embodiments may provide a system and methodfor estimating the effectiveness of one or more autonomous orsemi-autonomous vehicle technologies, functionalities, systems, and/orpieces of equipment on reducing a likelihood, and/or severity, of avehicle accident, such as depicted in FIG. 14 (discussed further below).The system and method, for each autonomous or semi-autonomous vehicletechnology or functionality that is analyzed, may evaluate or utilizethe effect or impact of one or more accident-related factors or elementson the effectiveness of the respective autonomous or semi-autonomousvehicle technology or functionality. The analysis or evaluation maydetermine the impact of each factor or element on how well an autonomousor semi-autonomous vehicle technology or functionality actually performsunder certain conditions (such as driving, vehicle, and/or roadconditions; accident or vehicle type; and/or other factors).

A. Technologies and Functionalities

Noted above, a system and method may analyze and/or evaluate theeffectiveness of autonomous or semi-autonomous vehicle technology,functionality, systems, and/or equipment. Individual technologies,functionalities, systems, and/or pieces of equipment may be evaluatedthat are related to: (1) fully autonomous (or driverless) vehicles; (2)limited driver control; (3) automatic or automated steering,acceleration, and/or braking; (4) blind spot monitoring; (5) collisionwarning; (6) adaptive cruise control; (7) parking assistance; (8) driveracuity or alertness monitoring; (9) pedestrian detection; (10) softwaresecurity for smart vehicles; (11) theft prevention; (12) artificialintelligence upgrades or updates; (13) GPS functionality; (14)vehicle-to-vehicle wireless communication; (15)vehicle-to-infrastructure one or two-way wireless communication; and/orother technology and functionality, including that discussed elsewhereherein. Each technology or functionality, and/or the accident avoidanceand/or mitigation effectiveness thereof, may be analyzed individuallyand/or in combination with one or more other technologies orfunctionalities.

B. Factors or Elements Impacting Effectiveness

The analysis and/or evaluation of the effectiveness of each technologyor functionality may determine an impact of one or more factors orelements that may degrade the performance of each autonomous orsemi-autonomous vehicle technology or functionality. For instance, eachfactor or element may lower or limit the effectiveness of an autonomousor semi-autonomous vehicle technology or functionality with respect toaccident avoidance and/or limiting the severity of vehicle accidents.The factors or elements may be analyzed individually and/or incombination with one or more other factors or elements.

Mentioned above, accident-related or other factors, elements, and/orconditions may impact the effectiveness of an autonomous orsemi-autonomous vehicle technology or functionality. The factors,elements, and/or conditions that may be evaluated may include: (1) pointof vehicle impact during a vehicle accident; (2) type of road that anaccident occurs on; (3) time of day that an accident occurs at; (4)weather conditions associated with an accident; (5) type of trip duringwhich the accident occurred (short, long, etc.); (6) vehicle style forthe vehicle(s) involved in an accident; (7) whether the vehiclesinvolved in the accident were equipped with vehicle-to-vehicle wirelesscommunication functionality; (8) whether the vehicle(s) involved in theaccident were equipped with vehicle-to-infrastructure orinfrastructure-to-vehicle wireless communication functionality; and/orother factors, elements, and/or conditions associated with, orimpacting, individual vehicle accidents.

An evaluation of the foregoing factors, elements, and/or conditions withrespect to multiple vehicle accidents involving vehicles having one ormore autonomous or semi-autonomous vehicle technologies orfunctionalities may indicate or suggest: (a) an overall effectivenessfor each individual autonomous or semi-autonomous vehicle technology orfunctionality, and/or (b) the impact (whether negative or positive) ofeach factor or element (type of road; type of vehicle; time of day;weather conditions; type of vehicle crash, i.e., point of impact; etc.)on the effectiveness of each autonomous or semi-autonomous vehicletechnology, functionality, and/or associated equipment. After which,insurance premiums, rates, discounts, rewards, points, and/or otherinsurance-related items for vehicles having one or more autonomous orsemi-autonomous vehicle technologies or functionalities may begenerated, adjusted, and/or updated.

C. Applying Driver Characteristics to Auto Insurance

Characteristics and/or driving behaviors of individual drivers orcustomers may also be used to estimate, generate, and/or adjustinsurance premiums, rates, discounts, rewards, and/or otherinsurance-related items for vehicles having one or more autonomous orsemi-autonomous vehicle technologies or functionalities. Drivercharacteristics and/or driver behavior, as well as driver location orhome address, may be compared, or analyzed in conjunction, with thefactors or elements that may impact the accident avoidance or mitigationeffectiveness of each autonomous or semi-autonomous vehicle technologyor functionality.

For instance, a driver or insured may mainly drive on the highway,during daylight hours, and/or primarily for short commutes to and fromwork. The driver or insured's vehicle may have certain autonomous orsemi-autonomous vehicle technologies or functionalities that have beenestablished to decrease the likelihood of an accident, the severity ofany accident, and/or otherwise increase safety or vehicle performanceduring highway, daylight, and/or short commute driving. If so, theinsurance rate, premium, discount, and/or another insurance-related itemfor the driver or insured may be adjusted in accordance with theestimated lower risk (of accident, and/or severe accident).

As one example, the impact of one factor (point of vehicle impact) onthe effectiveness of accident avoidance and/or mitigation for anautonomous or semi-autonomous vehicle technology or functionality may bedetermined. For instance, the impact of head-on collisions on theaccident avoidance and/or mitigation effectiveness of automatic brakingand/or automatic steering functionality may be analyzed. Also analyzedmay be the effect of point of vehicle impact on the accident avoidanceand/or mitigation effectiveness of automatic acceleration functionality.The impact of point of vehicle impact on the accident avoidance and/ormitigation effectiveness of other autonomous or semi-autonomoustechnologies and/or functionalities, including those discussed elsewhereherein, may additionally or alternatively be evaluated.

As another example, the impact of another factor (vehicle size or type)on the effectiveness of accident avoidance and/or mitigation for anautonomous or semi-autonomous vehicle technology, functionality, system,and/or piece of equipment may be determined. For instance, the impact ofthe vehicle being a compact car, mid-sized car, truck, SUV (sportutility vehicle), etc. on the accident avoidance and/or mitigationeffectiveness for blind spot monitoring functionality and/or driveracuity monitoring functionality may be analyzed. The impact of vehiclesize or type on the accident avoidance and/or mitigation effectivenessof other autonomous or semi-autonomous technologies and/orfunctionalities, including those discussed elsewhere herein, mayadditionally or alternatively be evaluated.

As a further example, the impact of another factor (type of road) on theeffectiveness of accident avoidance and/or mitigation for an autonomousor semi-autonomous vehicle technology, functionality, system, and/orpiece of equipment may be determined. For instance, the impact of thetype of road (whether a freeway, highway, toll way, rural road ortwo-lane state or county highway, and/or downtown or city street) on theaccident avoidance and/or mitigation effectiveness for adaptive cruisecontrol functionality and/or vehicle-to-vehicle functionality may beanalyzed. The impact of type of road on the accident avoidance and/ormitigation effectiveness of other autonomous or semi-autonomoustechnologies and/or functionalities, including those discussed elsewhereherein, may additionally or alternatively be evaluated.

Additionally, the amount of time or percentage of vehicle usage that anautonomous or semi-autonomous vehicle technology, functionality, system,and/or piece of equipment is used by the driver or vehicle operator maybe determined from sensor or smart vehicle data. Technology usageinformation gathered or collected may be used to generate, update,and/or adjust insurance policies, premiums, rates, discounts, rewards,points, programs, and/or other insurance-related items.

D. Exemplary System Overview

At a broad level, the methods and systems described herein may be viewedas combining information regarding autonomous (and/or semi-autonomous)vehicle operation technology with information regarding environmental orusage elements to evaluate one or more autonomous (and/orsemi-autonomous) operation features, determine one or more risk factorsfor the autonomous (and/or semi-autonomous) operation features, anddetermine vehicle insurance premiums based upon the risk factors. Insome embodiments, the autonomous operation features may include anautonomous driving software package or artificial intelligence foroperating an automobile. Evaluation of the autonomous operation featuresmay include evaluating both software and hardware associated with thefeatures in a test environment, as well as evaluating actual lossexperience associated with vehicles using the features in ordinaryoperation (i.e., operation not in a test environment). The risk factorsmay be associated with the relative ability of the autonomous operationfeatures to make control decisions that avoid accidents and othercollisions. The risk factors may be included in determining insurancepolicy premiums, which may in some embodiments include other factorsrelevant to the determination of the total risk associated with one ormore types of insurance coverage for an autonomous vehicle.

FIG. 14 illustrates a high-level flow diagram of an exemplary autonomous(and/or semi-autonomous) automobile insurance pricing system.Information regarding one or more autonomous operation featuretechnologies is collected, accessed, or otherwise received at block1402. Such information may relate to one or more of the followingtechnologies: a fully autonomous (driverless) vehicle operatingtechnology, a limited driver control technology, an automatic steeringtechnology, an automatic acceleration and/or braking technology, a blindspot monitoring and/or other information augmenting technology, acollision and/or other warning technology, an adaptive cruise controltechnology, a parking assist technology, and/or other autonomousoperation technologies (including those described elsewhere herein orlater developed). The autonomous operation feature technologies of block1402 may be associated with one or more environmental or usage elements,information regarding which may be collected, accessed, or otherwisereceived at block 1404. Such information may relate to one or more ofthe following elements: a point of impact between the autonomousautomobile and another object (e.g., another vehicle, an infrastructurecomponent, or another moving or fixed object within the autonomousautomobile's environment), a type of road (e.g., a limited accesshighway, a residential neighborhood street, or a main thoroughfare), atime of day and/or date (e.g., rush hour, weekend, or holiday), aweather condition (e.g., light levels, cloud cover, precipitation,temperature, wind, or ground cover such as ice or snow), a type and/orpurpose of vehicle trip (e.g., commuting, interstate travel, orleisure), a vehicle style and/or type, a vehicle-to-vehiclecommunication, or a vehicle-to-infrastructure communication. Theinformation regarding the elements in block 1404 may be furtherassociated with the information regarding the technology in block 1402.Some technologies may be adapted to utilize information regarding someelements, and some elements may be more relevant to some technologiesthan to others.

The information regarding the technologies and elements may then be usedin evaluating the performance of the autonomous (and/or semi-autonomous)operation features. The performance or sophistication of the autonomousoperating features (e.g., autonomous driving software or artificialintelligence) may be determined within a test environment at block 1406,as described above. The evaluation may include a variety of combinationsof technologies and elements, and one or more risk levels or riskprofiles may be determined as part of or based upon the evaluation. Insome embodiments, the evaluation may include testing the autonomousoperation features on a test track or other test facility by installingthe features within a test automobile. The test performance may then besupplemented or compared with actual loss experience informationrelating to the autonomous operating features in actual drivingsituations recorded at block 1408. The recorded actual loss experiencefrom block 1408 and/or the evaluated test performance from block 1406may be used to determine a relative or total risk factor for theautonomous operation features based upon the observed or expectedability of the autonomous operation features to make driving decisionsfor the autonomous automobile and avoid crashes, collisions, or otherlosses at block 1410. Based upon the risk factor determined at block1410, one or more premiums or components of premiums for an automobileinsurance policy may be determined at block 1412, as discussed above.These premiums make take into account the risks associated withautonomous operation features or combinations of features, as well asexpected environmental or usage conditions, factors, or levels. Thepremiums determined at block 1412 may then be presented to a customer orpotential customer for review, selection, or acceptance and purchase.

Exemplary Methods of Evaluating Impact on Effectiveness

In one aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include: (a) evaluating, via aprocessor, a vehicle accident avoidance and/or mitigation effectivenessof, and/or associated with, an autonomous or semi-autonomous vehicletechnology, functionality, system, and/or piece of equipment underreal-world driving conditions, the real-world driving conditionsincluding one or more conditions that effect or impact the likelihood,and/or severity, of a vehicle accident or collision; (b) updating,adjusting, and/or generating an auto insurance policy, premium, rate,and/or discount, via the processor, based upon the accident avoidanceand/or mitigation effectiveness of the autonomous or semi-autonomousvehicle technology, functionality, system, or equipment for a vehicleequipped with the autonomous or semi-autonomous vehicle feature,technology, system, and/or piece of equipment; (c) presenting (all or aportion of) the updated, adjusted, and/or generated auto insurancepolicy, premium, rate, and/or discount to an insured, driver, or ownerof the vehicle equipped with the autonomous or semi-autonomous vehicletechnology, functionality, system, and/or piece of equipment for theirreview, approval, and/or modification on a display screen associatedwith a computing device; (d) receiving, via the processor, an approvalof and/or modification to the auto insurance policy, premium, rate,and/or discount from the insured, driver, or owner of the vehicleequipped with the autonomous or semi-autonomous vehicle technology,functionality, system, and/or piece of equipment; and/or (e) updating anauto insurance policy, premium, rate, and/or discount for, and/or thenadjusting appropriate amounts to be charged to, the insured, driver, orowner of the vehicle equipped with the autonomous or semi-autonomousvehicle technology, functionality, system, and/or piece of equipmentbased upon the information received from the insured, driver, or ownerof the vehicle equipped with the autonomous or semi-autonomous vehicletechnology, functionality, system, and/or piece of equipment.

The step of (a) evaluating, via the processor, an accident avoidance ormitigation effectiveness of, or associated with, an autonomous orsemi-autonomous vehicle technology, functionality, system, or piece ofequipment under real-world driving conditions may include: (i) analysisof a plurality of vehicle accidents involving one or more vehicleshaving the autonomous or semi-autonomous vehicle technology,functionality, system, or piece of equipment, and/or (ii) testingvehicles equipped with the autonomous or semi-autonomous vehicletechnology, functionality, system, and/or piece of equipment underreal-world conditions and gathering data. The method may includeadditional, fewer, or alternate actions, including those discussedelsewhere herein.

In another aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include: (1) updating, adjusting, and/orgenerating an auto insurance policy, premium, rate, and/or discount, viaa processor, based upon and/or taking into consideration: (a) one ormore autonomous or semi-autonomous vehicle features, technologies,systems, and/or pieces of equipment; (b) conditions and/or factorsimpacting the effectiveness of each autonomous or semi-autonomousvehicle feature, technology, system, and/or piece of equipment withrespect to accident avoidance and/or mitigation; (c) driver or insuredactual characteristics or driving behavior, and/or geographical locationassociated with the driver, insured, or vehicle; and/or (d) driver orinsured actual usage of the one or more autonomous or semi-autonomousvehicle features, technologies, systems, and/or pieces of equipment; (2)presenting on a display (such on a display of a computing deviceassociated with the driver or insured, or a sales agent), all or aportion of, the updated, adjusted, and/or generated auto insurancepolicy, premium, rate, and/or discount for the driver's or insured'sreview, approval, and/or modification; (3) receiving and/or acceptingthe approval and/or modification via wireless communication from thecomputing device associated with the driver or insured at the processor;and/or (4) processing, handling, and/or updating the auto insurancepolicy accordingly and/or billing the driver or insured appropriately(via the processor) for the updated or new auto insurance coverage.

The one or more autonomous or semi-autonomous vehicle features,technologies, systems, and/or pieces of equipment may be or include anupdated or revised version of computer or processing instructionsrelated to the one or more autonomous or semi-autonomous vehiclefeatures, technologies, systems, and/or pieces of equipment. The methodmay include additional, fewer, or alternate actions, including thosediscussed elsewhere herein.

Exemplary Methods of Applying Auto Insurance Risk Factors

In another aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include: (i) estimating a risk factorassociated with auto insurance, via a processor, based upon (1) one ormore autonomous or semi-autonomous vehicle technologies,functionalities, systems, and/or pieces of equipment; and/or (2) one ormore accident-related conditions or factors that impact theeffectiveness of the one or more autonomous or semi-autonomous vehicletechnologies, functionalities, systems, and/or pieces of equipment,individually and/or as a group or collectively; (ii) generating,updating, and/or adjusting an auto insurance policy, premium, rate,and/or discount, via the processor, based upon the (a) risk factorassociated with auto insurance estimated, and/or (b) actual drivingcharacteristics and/or behaviors (such as typical drivingpatterns/paths/routes, geographical location, type of trips usuallytaken, etc.) of a driver, an insured, or an owner of a vehicle withhaving the one or more autonomous or semi-autonomous vehicletechnologies, functionalities, systems, and/or pieces of equipment;(iii) presenting (all or a portion of) the auto insurance policy,premium, rate, and/or discount generated, updated, and/or adjusted,under the direction and/or control of the processor, on a display of acomputing device (such as a mobile device associated with the driver,insured, or vehicle owner or an insurance representative) for theirreview, approval, and/or modification; (iv) receiving, at the processor,such as via wireless communication from the computing device, the autoinsurance policy, premium, rate, and/or discount approved by the driver,insured, or vehicle owner; and/or (v) processing, handling, and/orupdating the new auto insurance policy, premium, rate, and/or discountvia the processor such that the customer (driver, insured, vehicle owneror operator) is billed appropriately for the amount of auto insurancecoverage agreed upon or purchased.

An amount or percentage of driving time that the driver or insured usesthe one or more autonomous or semi-autonomous vehicle technologies,functionalities, systems, or pieces of equipment while driving thevehicle may be used to update, adjust, and/or generate the insurancepolicy, premium, rate, and/or discount. The method may includeadditional, fewer, or alternate actions, including those discussedelsewhere herein.

In another aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include: (1) applying an accident riskfactor associated with one or more autonomous or semi-autonomous vehicletechnologies, functionalities, systems, and/or pieces of equipment to anauto insurance policy, premium, rate, discount, reward, etc. for adriver or insured's vehicle having, or equipped with, the one or moreautonomous or semi-autonomous vehicle technologies, functionalities,systems, and/or pieces of equipment (via a processor), the risk factorbeing generated from evaluation of one or more driving and/oraccident-related conditions impacting an effectiveness of the one ormore autonomous or semi-autonomous vehicle technologies,functionalities, systems, and/or pieces of equipment with respect toaccident avoidance and/or mitigation; (2) presenting (under thedirection and/or control of the processor) on a display of a computingdevice (such as a mobile device associated with the driver or insured,or a sales agent) (all and/or portions of) the auto insurance policy,premium, rate, discount, reward, etc. to which the accident risk factorwas applied for the driver's or insured's review, approval, and/ormodification; (3) receiving and/or accepting the approved and/ormodified auto insurance policy, premium, rate, discount, reward, etc. atthe processor (such as via wireless communication from the computingdevice); and/or (4) processing, handling, and/or updating (via theprocessor) the auto insurance policy for the driver's or insured'svehicle having, or equipped with, the one or more autonomous orsemi-autonomous vehicle technologies, functionalities, systems, and/orpieces of equipment accordingly based upon the information received tobill the customer (e.g., driver, insured, or vehicle owner/operator) anappropriate amount for the amount of auto insurance coverage purchasedand/or agreed to. The method may include additional, fewer, or alternateactions, including those discussed elsewhere herein.

Exemplary Methods of Evaluating Artificial Intelligence

In another aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include (1) determining, via aprocessor, the automobile accident avoidance and/or mitigation relatedeffectiveness associated with, or for, a revision or update of computeror processor instructions that direct and/or control one or moreautonomous or semi-autonomous vehicle technologies, functionalities,systems, and/or pieces of equipment (and that may be stored on anon-transitory computer readable media or medium), the effectivenessdetermination taking into consideration: (a) actual vehicle accidentinformation for accidents involving vehicles equipped with the revisionor update of the computer or processor instructions that direct and/orcontrol the one or more autonomous or semi-autonomous vehicletechnologies, functionalities, systems, and/or pieces of equipment;and/or (b) physical testing of vehicles equipped with the revision orupdate of the computer or processor instructions that direct and/orcontrol the one or more autonomous or semi-autonomous vehicletechnologies, functionalities, systems, and/or pieces of equipment. Themethod may also include (2) updating, adjusting, and/or generating aninsurance policy, premium, rate, and/or discount for a vehicle equippedwith the revision or update of computer or processor instructions thatdirect and/or control the one or more autonomous or semi-autonomousvehicle technologies, functionalities, systems, and/or pieces ofequipment. The method may include additional, fewer, or alternateactions, including those discussed elsewhere herein.

In another aspect, a computer-implemented method of updating, adjusting,and/or generating an insurance policy, premium, rate, and/or discountmay be provided. The method may include: (1) testing an upgrade orupdate to computer or processor instructions that direct and/or controlone or more autonomous or semi-autonomous vehicle technologies,functionalities, systems, and/or pieces of equipment (and that arestored on a non-transitory computer readable media or medium); (2)determining an increase in accident avoidance or mitigationeffectiveness based upon the upgraded or updated computer or processorinstructions that direct and/or control the one or more autonomous orsemi-autonomous vehicle technologies, functionalities, systems, and/orpieces of equipment; and/or (3) updating, adjusting, and/or generatingan insurance policy, premium, rate, and/or discount for a vehicleequipped with the upgraded or updated computer or processor instructionsthat direct and/or control the one or more autonomous or semi-autonomousvehicle technologies, functionalities, systems, and/or pieces ofequipment based upon the increase in accident avoidance or mitigationeffectiveness determined. The method may include additional, fewer, oralternate actions, including those discussed elsewhere herein.

Additional Exemplary Methods

In one aspect, a computer-implemented method of evaluating risk ofautonomous or semi-autonomous vehicle technology may be provided. Themethod may include (1) generating, via one or more processors, a virtualtest scenario, the virtual test scenario including one or moreaccident-related factors or conditions; (2) applying, via the one ormore processors, the virtual test scenario to a package of computerinstructions that instruct a vehicle processor to perform an autonomousor semi-autonomous functionality; (3) analyzing, via the one or moreprocessors, a performance of the autonomous or semi-autonomousfunctionality under virtual conditions associated with the virtual testscenario; (4) determining, via the one or more processors, aninsurance-based risk (e.g., a risk of a vehicle accident) of, orassociated with, the package of computer instructions that instruct thevehicle processor to perform the autonomous or semi-autonomousfunctionality; and/or (5) generating, updating, or adjusting, via theone or more processors, a premium, rate, discount, reward, or otherinsurance item associated with an insurance policy for an autonomous orsemi-autonomous vehicle employing the package of computer instructionsthat instruct the vehicle processor to perform the autonomous orsemi-autonomous functionality based upon the insurance-based risk of, orassociated with, the package of computer instructions. The method mayinclude additional, fewer, or alternate actions, including thosediscussed elsewhere herein.

For instance, the one or more accident-related factors or conditions ofthe virtual test scenario may include road, construction, traffic, othervehicle, and/or weather factors or conditions. The virtual test scenariomay include a virtual simulation of virtual traffic traveling on avirtual road, and each virtual vehicle traveling on a virtual route at avirtual speed. Determining, via the one or more processors, theinsurance-based risk (e.g., a risk of a vehicle accident) of, orassociated with, the package of computer instructions may includedetermining whether the package of computer instructions made a corrector proper decision given the road, construction, traffic, other vehicle,and/or weather conditions of the virtual test scenario.

In another aspect, a computer-implemented method of evaluating risk ofautonomous or semi-autonomous vehicle technology may be provided. Themethod may include (1) collecting or receiving, via one or moreprocessors, a virtual log of actual or real-world driving performance ofautonomous or semi-autonomous functionality from a vehicle (or smartvehicle controller); (2) determining, via the one or more processors, aninsurance-based risk (e.g., a risk of a vehicle accident) of, orassociated with, the autonomous or semi-autonomous functionality fromanalysis of the virtual log of actual or real-world driving performanceof the autonomous or semi-autonomous functionality, respectively; and/or(3) generating, updating, or adjusting, via the one or more processors,a premium, rate, discount, reward, or other insurance item associatedwith an insurance policy for an autonomous or semi-autonomous vehicleemploying the autonomous or semi-autonomous functionality based upon theinsurance-based risk of, or associated with, the autonomous orsemi-autonomous functionality. The method may include additional, less,or alternate functionality, including that discussed elsewhere herein.

For instance, determining, via the one or more processors, theinsurance-based risk (e.g., a risk of a vehicle accident) of, orassociated with, the autonomous or semi-autonomous functionality fromanalysis of the virtual log of actual or real-world driving performancemay include analysis of: (1) decisions that the autonomous orsemi-autonomous functionality, or associated computer instructions,made; and/or (2) the environment and/or operating conditions (e.g.,road, construction, traffic, and/or weather conditions) under which thedecisions were made. Additionally or alternatively, determining, via theone or more processors, the insurance-based risk (e.g., a risk of avehicle accident) of, or associated with, the autonomous orsemi-autonomous functionality from analysis of the virtual log of actualor real-world driving performance may include analysis of: (1) evasivemaneuvers that the autonomous or semi-autonomous functionality, orassociated computer instructions, made (or directed the vehicle toperform); and/or (2) the environment and/or operating conditions (e.g.,road, construction, traffic, and/or weather conditions) under which theevasive maneuvers were made. Further, determining, via the one or moreprocessors, the insurance-based risk (e.g., a risk of a vehicleaccident) of, or associated with, the autonomous or semi-autonomousfunctionality from analysis of the virtual log of actual or real-worlddriving performance may include analysis of (1) how the autonomous orsemi-autonomous functionality, or associated computer instructions,respond to other drivers on the road maneuvering or changing speed;and/or (2) grading, rating, or otherwise evaluating the responsivenessof the autonomous or semi-autonomous functionality, or associatedcomputer instructions, to other drivers' driving behavior.

In another aspect, a computer-implemented method of evaluating risk ofautonomous or semi-autonomous vehicle technology and/or adjustingautonomous or semi-autonomous vehicle technology may be provided. Themethod may include (1) determining, via one or more processors, anoptimum setting for an autonomous or semi-autonomous vehicle technology;(2) detecting, via the one or more processors, that an actual settingfor an autonomous or semi-autonomous vehicle system of a vehicle isdifferent than the optimum setting for the autonomous or semi-autonomousvehicle technology; (3) generating a recommendation, via the one or moreprocessors, to change the actual setting for the autonomous orsemi-autonomous vehicle system to the optimum setting; and/or (4)causing the recommendation, via the one or more processors, to bepresented to a driver of the vehicle having the autonomous orsemi-autonomous vehicle system to facilitate the driver changing theactual setting to the optimum setting (or otherwise accepting, or beingnotified of, an automatic change to the optimum setting). The method mayinclude additional, less, or alternate functionality, including thatdiscussed elsewhere herein.

For instance, the method may include generating or adjusting, via theone or more processors, a premium, rate, discount, or reward of an autoinsurance policy for the vehicle based upon the vehicle having or beingequipped with the recommendation functionality associated withrecommending optimum settings, and/or a percentage of the driveraccepting the recommendations provided. The optimum setting that isdetermined may be determined based upon a setting for the autonomous orsemi-autonomous vehicle technology that reduces a likelihood of thevehicle employing the technology having, or being involved in, a vehicleaccident or collision. Additionally or alternatively, the optimumsetting that is determined may be determined based upon risk associatedwith the autonomous or semi-autonomous vehicle technology.

In another aspect, a computer-implemented method of determining accidentfault may be provided. The method may include (1) receiving orcollecting, at or via a remote server (or processor) associated with aninsurance provider, performance data associated with autonomous orsemi-autonomous vehicle technology for an insured vehicle involved in avehicle accident, the insured vehicle being insured by an insurancepolicy issued by the insurance provider; (2) analyzing, at or via theremote server, the performance data received; (3) determining, at or viathe remote server, from analysis of the performance data: (a) a firstpercentage of fault of the vehicle accident for the autonomous orsemi-autonomous vehicle technology of the insured vehicle in operationat a time of the vehicle accident; and/or (b) a second percentage offault of the vehicle accident for an insured driver who was driving theinsured vehicle during the vehicle accident; and/or (4) adjusting orupdating, at or via the remote server, a premium, rate, discount, orreward of the insurance policy covering the insured vehicle based uponthe first and second percentages of fault that are assigned to theinsured vehicle and insured driver, respectively, from the analysis ofthe performance data. The method may include additional, less, oralternate functionality, including that discussed elsewhere herein.

For instance, the method may include further determining, at or via theremote server, from analysis of the performance data: (c) a thirdpercentage of fault of the accident for other vehicles or driversinvolved in and/or causing the vehicle accident. The method may includehandling, at or via the remote server, an insurance claim for thevehicle accident submitted by the insured using, or based upon, theperformance data received associated with the autonomous orsemi-autonomous vehicle functionality exhibited during the vehicleaccident.

The method may include changing, at or via the remote server, policycoverages (and/or premiums, rates, discounts, etc.) for the insured orthe insured vehicle using, or based upon, the performance data receivedassociated with the autonomous or semi-autonomous vehicle functionalityexhibited during the vehicle accident. Additionally or alternatively,the method may include changing, at or via the remote server, liabilitylimits or coverages (and/or premiums, rates, discounts, etc.) for theinsured or the insured vehicle using, or based upon, the performancedata received associated with the autonomous or semi-autonomous vehiclefunctionality exhibited during the vehicle accident.

While the preferred embodiments of the invention have been described, itshould be understood that the invention is not so limited andmodifications may be made without departing from the invention. Thescope of the invention is defined by the appended claims, and alldevices that come within the meaning of the claims, either literally orby equivalence, are intended to be embraced therein.

It is therefore intended that the foregoing detailed description beregarded as illustrative rather than limiting, and that it be understoodthat it is the following claims, including all equivalents, that areintended to define the spirit and scope of this invention.

Other Matters

Although the text herein sets forth a detailed description of numerousdifferent embodiments, it should be understood that the legal scope ofthe invention is defined by the words of the claims set forth at the endof this patent. The detailed description is to be construed as exemplaryonly and does not describe every possible embodiment, as describingevery possible embodiment would be impractical, if not impossible. Onecould implement numerous alternate embodiments, using either currenttechnology or technology developed after the filing date of this patent,which would still fall within the scope of the claims.

It should also be understood that, unless a term is expressly defined inthis patent using the sentence “As used herein, the term ‘______’ ishereby defined to mean . . . ” or a similar sentence, there is no intentto limit the meaning of that term, either expressly or by implication,beyond its plain or ordinary meaning, and such term should not beinterpreted to be limited in scope based upon any statement made in anysection of this patent (other than the language of the claims). To theextent that any term recited in the claims at the end of this disclosureis referred to in this disclosure in a manner consistent with a singlemeaning, that is done for sake of clarity only so as to not confuse thereader, and it is not intended that such claim term be limited, byimplication or otherwise, to that single meaning. Finally, unless aclaim element is defined by reciting the word “means” and a functionwithout the recital of any structure, it is not intended that the scopeof any claim element be interpreted based upon the application of 35U.S.C. § 112(f).

Throughout this specification, plural instances may implementcomponents, operations, or structures described as a single instance.Although individual operations of one or more methods are illustratedand described as separate operations, one or more of the individualoperations may be performed concurrently, and nothing requires that theoperations be performed in the order illustrated. Structures andfunctionality presented as separate components in example configurationsmay be implemented as a combined structure or component. Similarly,structures and functionality presented as a single component may beimplemented as separate components. These and other variations,modifications, additions, and improvements fall within the scope of thesubject matter herein.

Additionally, certain embodiments are described herein as includinglogic or a number of routines, subroutines, applications, orinstructions. These may constitute either software (code embodied on anon-transitory, tangible machine-readable medium) or hardware. Inhardware, the routines, etc., are tangible units capable of performingcertain operations and may be configured or arranged in a certainmanner. In example embodiments, one or more computer systems (e.g., astandalone, client or server computer system) or one or more hardwaremodules of a computer system (e.g., a processor or a group ofprocessors) may be configured by software (e.g., an application orapplication portion) as a hardware module that operates to performcertain operations as described herein.

In various embodiments, a hardware module may be implementedmechanically or electronically. For example, a hardware module maycomprise dedicated circuitry or logic that is permanently configured(e.g., as a special-purpose processor, such as a field programmable gatearray (FPGA) or an application-specific integrated circuit (ASIC) toperform certain operations. A hardware module may also compriseprogrammable logic or circuitry (e.g., as encompassed within ageneral-purpose processor or other programmable processor) that istemporarily configured by software to perform certain operations. Itwill be appreciated that the decision to implement a hardware modulemechanically, in dedicated and permanently configured circuitry, or intemporarily configured circuitry (e.g., configured by software) may bedriven by cost and time considerations.

Accordingly, the term “hardware” should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which hardwareis temporarily configured (e.g., programmed), the hardware need not beconfigured or instantiated at any one instance in time. Software mayaccordingly configure a processor, for example, to constitute aparticular hardware module at one instance of time and to constitute adifferent hardware module at a different instance of time. Hardwareelements can provide information to, and receive information from, otherhardware elements. Accordingly, the described hardware may be regardedas being communicatively coupled.

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules. Similarly, the methods or routinesdescribed herein may be at least partially processor-implemented. Theperformance of certain of the operations may be distributed among theone or more processors, not only residing within a single machine, butdeployed across a number of machines. In some example embodiments, theprocessor or processors may be located in a single location (e.g.,within a home environment, an office environment or as a server farm),while in other embodiments the processors may be distributed across anumber of locations.

Unless specifically stated otherwise, discussions herein using wordssuch as “processing,” “computing,” “calculating,” “determining,”“presenting,” “displaying,” or the like may refer to actions orprocesses of a machine (e.g., a computer) that manipulates or transformsdata represented as physical (e.g., electronic, magnetic, or optical)quantities within one or more memories (e.g., volatile memory,non-volatile memory, or a combination thereof), registers, or othermachine components that receive, store, transmit, or displayinformation. As used herein any reference to “one embodiment” or “anembodiment” means that a particular element, feature, structure, orcharacteristic described in connection with the embodiment may beincluded in at least one embodiment. The appearances of the phrase “inone embodiment” in various places in the specification are notnecessarily all referring to the same embodiment. Some embodiments maybe described using the expression “coupled” and “connected” along withtheir derivatives. For example, some embodiments may be described usingthe term “coupled” to indicate that two or more elements are in directphysical or electrical contact. The term “coupled,” however, may alsomean that two or more elements are not in direct contact with eachother, but yet still co-operate or interact with each other. Theembodiments are not limited in this context.

As used herein, the terms “comprises,” “comprising,” “includes,”“including,” “has,” “having” or any other variation thereof, areintended to cover a non-exclusive inclusion. For example, a process,method, article, or apparatus that comprises a list of elements is notnecessarily limited to only those elements but may include otherelements not expressly listed or inherent to such process, method,article, or apparatus. Further, unless expressly stated to the contrary,“or” refers to an inclusive or and not to an exclusive or. In addition,use of the “a” or “an” are employed to describe elements and componentsof the embodiments herein. This is done merely for convenience and togive a general sense of the description. In this description, and theclaims that follow, the singular also includes the plural unless it isobvious that it is meant otherwise. This detailed description is to beconstrued as exemplary only and does not describe every possibleembodiment, as describing every possible embodiment would beimpractical, if not impossible. One could implement numerous alternateembodiments, using either current technology or technology developedafter the filing date of this application.

Upon reading this disclosure, those of skill in the art will appreciatestill additional alternative structural and functional designs forsystem and a method for assigning mobile device data to a vehiclethrough the disclosed principles herein. Thus, while particularembodiments and applications have been illustrated and described, it isto be understood that the disclosed embodiments are not limited to theprecise construction and components disclosed herein. Variousmodifications, changes and variations, which will be apparent to thoseskilled in the art, may be made in the arrangement, operation anddetails of the method and apparatus disclosed herein without departingfrom the spirit and scope defined in the appended claims.

The particular features, structures, or characteristics of any specificembodiment may be combined in any suitable manner and in any suitablecombination with one or more other embodiments, including the use ofselected features without corresponding use of other features. Inaddition, many modifications may be made to adapt a particularapplication, situation or material to the essential scope and spirit ofthe present invention. It is to be understood that other variations andmodifications of the embodiments of the present invention described andillustrated herein are possible in light of the teachings herein and areto be considered part of the spirit and scope of the present invention.

What is claimed is:
 1. A computer-implemented method of evaluatingautonomous or semi-autonomous vehicle technology, the method comprising:generating, by one or more processors of a computer system configured tovirtually test responses of computer instructions associated withautonomous or semi-autonomous vehicle technology within a virtualenvironment, a virtual test scenario related to vehicle operation;applying, by the one or more processors of the computer system, thevirtual test scenario to a package of computer instructions configuredto instruct a processor of an autonomous or semi-autonomous vehicle toperform an autonomous or semi-autonomous functionality; determining, bythe one or more processors of the computer system, one or more responsesof the package of computer instructions to the applied virtual testscenario; and determining, by the one or more processors of the computersystem, a performance metric associated with the package of computerinstructions based at least in part upon the one or more responses ofthe package of computer instructions to the applied virtual testscenario, wherein the performance metric is associated with a risk levelassociated with use of the package of computer instructions in vehicleoperation.
 2. The method of claim 1, wherein the virtual test scenarioincludes virtual sensor data representative of one or more conditionsassociated with an increased risk of a vehicle accident.
 3. The methodof claim 2, wherein the one or more conditions of the virtual testscenario include one or more of the following: road conditions,construction, traffic conditions, actions of other vehicles, or weatherconditions.
 4. The method of claim 2, further comprising: determining,by one or more processors, the risk level associated with the package ofcomputer instructions based upon the determined performance metric. 5.The method of claim 4, wherein determining the risk level associatedwith the package of computer instructions includes determining whetherthe package of computer instructions made a correct decision given theone or more conditions of the virtual test scenario.
 6. The method ofclaim 1, wherein the virtual test scenario includes a virtual simulationof virtual traffic traveling on a virtual road, including a plurality ofvirtual vehicles.
 7. The method of claim 6, wherein the virtual testscenario further includes a virtual route and a virtual speed for eachof the plurality of virtual vehicles traveling on the virtual road. 8.The method of claim 1, wherein the risk level indicates a risk of thevehicle being involved in a vehicle accident.
 9. A computer system forevaluating autonomous or semi-autonomous vehicle technology, comprising:one or more processors; a program memory coupled to the one or moreprocessors and storing (i) a package of computer instructions configuredto instruct a processor of an autonomous or semi-autonomous vehicle toperform an autonomous or semi-autonomous functionality and (ii)executable instructions that, when executed by the one or moreprocessors, cause the computer system to virtually test responses of thepackage of computer instructions within a virtual environment by:generating a virtual test scenario related to vehicle operation;applying the virtual test scenario to the package of computerinstructions; determining one or more responses of the package ofcomputer instructions to the applied virtual test scenario; anddetermining a performance metric associated with the package of computerinstructions based at least in part upon the one or more responses ofthe package of computer instructions to the applied virtual testscenario, wherein the performance metric is associated with a risk levelassociated with use of the package of computer instructions in vehicleoperation.
 10. The computer system of claim 9, wherein the virtual testscenario includes virtual sensor data representative of one or moreconditions associated with an increased risk of a vehicle accident. 11.The computer system of claim 9, wherein the executable instructionsfurther cause the computer system to determine the risk level associatedwith the package of computer instructions based upon the determinedperformance metric by determining whether the package of computerinstructions made a correct decision given the one or more conditions ofthe virtual test scenario.
 12. The computer system of claim 9, whereinthe virtual test scenario includes a virtual simulation of virtualtraffic traveling on a virtual road, including a virtual route and avirtual speed for each of a plurality of virtual vehicles.
 13. Thecomputer system of claim 9, wherein the risk level indicates a risk ofthe vehicle being involved in a vehicle accident.
 14. A tangible,non-transitory computer-readable medium storing executable instructionsfor evaluating autonomous or semi-autonomous vehicle technology that,when executed by at least one processor of a computer system, cause thecomputer system to: access a package of computer instructions configuredto instruct a processor of an autonomous or semi-autonomous vehicle toperform an autonomous or semi-autonomous functionality; generate avirtual test scenario related to vehicle operation; apply the virtualtest scenario to the package of computer instructions; determine one ormore responses of the package of computer instructions to the appliedvirtual test scenario; and determine a performance metric associatedwith the package of computer instructions based at least in part uponthe one or more responses of the package of computer instructions to theapplied virtual test scenario, wherein the performance metric isassociated with a risk level associated with use of the package ofcomputer instructions in vehicle operation.
 15. The tangible,non-transitory computer-readable medium of claim 14, wherein the virtualtest scenario includes virtual sensor data representative of one or moreconditions associated with an increased risk of a vehicle accident. 16.The tangible, non-transitory computer-readable medium of claim 14,further storing executable instructions that cause the computer systemto determine the risk level associated with the package of computerinstructions based upon the determined performance metric by determiningwhether the package of computer instructions made a correct decisiongiven the one or more conditions of the virtual test scenario.
 17. Thetangible, non-transitory computer-readable medium of claim 14, whereinthe virtual test scenario includes a virtual simulation of virtualtraffic traveling on a virtual road, including a virtual route and avirtual speed for each of a plurality of virtual vehicles.
 18. Thetangible, non-transitory computer-readable medium of claim 14, whereinthe risk level indicates a risk of the vehicle being involved in avehicle accident.